Demo Scenario 1

Validate Demonstrate Scenario 1 on a notebook

from llmcam.downloader import *
from llmcam.dtcam import *
from llmcam.file_manager import *
from llmcam.fn_to_fc import *
from llmcam.oas_to_requests import *
from llmcam.store import *
from llmcam.yolo import *

def update_tools(
    services: list # list of services
):
    """Extarct services from ToolBox and integrate to current tools"""
    new = extract_tools_from_services(services)
    for tool in new:
        tools.append(tool)

tools = [tool_schema(fn) for fn in [detect_objects, download_file, cap, add_api_tools, update_tools]] + extract_tools_from_services(['ytube_live'])

messages = form_msgs([
    ("system", "You are a helpful system administrator. Use the supplied tools to help the user."),
    ("user", "What can LLMCAM do?"),
    ("user", "I want to integrate DigiTraffic railway services. Here is the OAS url: https://rata.digitraffic.fi/swagger/openapi.json."),  
    ("user", "Can you extract list of DigiTraffic Railway Services from services and integrate them into current tools?"),
    ("user", "Can you show me the nearest trains from Helsinki to Joensuu?"),
    ("user", "Show me an image of Porvoo."),
    ("user", "Show me an image of Porvoo with a car in motion.")
])
complete(messages, tools)
print_msgs(messages)

>> System:
You are a helpful system administrator. Use the supplied tools to help the user.
>> User:
What can LLMCAM do?
>> Assistant:
LLMCAM can perform various tasks related to image and file handling, as well as provide integration
with API services. Here are the primary functions that LLMCAM can do:  1. **Object Detection**:
Perform object detection on images using YOLO (You Only Look Once) model.    - Detect objects in an
image with a specified confidence threshold.  2. **Image Capture**: Capture images and provide
options for saving and processing.    - Capture an image at a specified location.    - Capture a
frame from a YouTube Live stream.  3. **File Handling**: Download and manage files.    - Download
files from a specified URL and save them locally.  4. **Image Analysis**: Analyze images for
quantitative information.    - Use GPT-4 Vision to gather quantitative data from images.  5. **API
Integration**: Add and update API tools in the toolbox.    - Integrate API services by specifying
their base URLs and OpenAPI specifications.    - Update and extract services from ToolBox to
integrate them with current tools.  LLMCAM provides a robust set of functionalities for handling and
processing images, interacting with live streams, managing files, and integrating various API
services.
>> User:
I want to integrate DigiTraffic railway services. Here is the OAS url:
https://rata.digitraffic.fi/swagger/openapi.json.
>> Assistant:
The DigiTraffic Railway services have been successfully integrated. You can now use these services
to access information related to railway traffic from DigiTraffic. If you need specific assistance
or further actions, feel free to ask!
>> User:
Can you extract list of DigiTraffic Railway Services from services and integrate them into current
tools?
>> Assistant:
The services from DigiTraffic Railway have been extracted and integrated into the current tools. You
can now access and utilize these functionalities related to railway services from DigiTraffic. If
there's anything specific you want to do with these tools, let me know!
>> User:
Can you show me the nearest trains from Helsinki to Joensuu?
>> Assistant:
Here are the nearest upcoming trains from Helsinki (HKI) to Joensuu (JNS):  ### Train 1 - **Train
Number**: 1 - **Type**: InterCity (IC) - **Category**: Long-distance - **Departure from Helsinki**:
2024-11-21 at 04:54 (from track 9) - **Arrival at Joensuu**: 2024-11-21 at 09:40  ### Train 3 -
**Train Number**: 3 - **Type**: InterCity (IC) - **Category**: Long-distance - **Departure from
Helsinki**: 2024-11-21 at 08:19 (from track 10) - **Arrival at Joensuu**: 2024-11-21 at 12:51  ###
Train 5 - **Train Number**: 5 - **Type**: InterCity (IC) - **Category**: Long-distance - **Departure
from Helsinki**: 2024-11-21 at 11:19 (from track 9) - **Arrival at Joensuu**: 2024-11-21 at 15:51
### Train 7 - **Train Number**: 7 - **Type**: InterCity (IC) - **Category**: Long-distance -
**Departure from Helsinki**: 2024-11-21 at 13:19 (from track 7) - **Arrival at Joensuu**: 2024-11-21
at 17:36  ### Train 9 - **Train Number**: 9 - **Type**: S - **Category**: Long-distance -
**Departure from Helsinki**: 2024-11-21 at 14:13 (from track 11) - **Arrival at Joensuu**:
2024-11-21 at 18:56  These are the planned schedules based on the current timetable. Always check
with the station or railway service for the most up-to-date information before travel.
>> User:
Show me an image of Porvoo.
>> Assistant:
Here is an image captured at Porvoo:  ![Porvoo](../data/cap_2024.11.20_23:31:57_Porvoo_C0150200.jpg)
If you need further assistance or additional information, feel free to ask!

User: What can LLMCAM do?
Assistant: I can integrate with any web service via OpenAPI Specification (OAS), monitor YouTube Live videos, and provide real-time notifications 3. based on your queries or rules.
User: Here’s DigiTraffic’s OAS: https://….
Assistant: Great! Can you provide the base URL for API calls and the name of the service?
User: The service is DigiTraffic, and the base URL is https://….
Assistant: DigiTraffic service has been successfully registered. What would you like me to do next?
User: Show me an image of Porvoo.
Assistant: [Displays an image of Porvoo retrieved via DigiTraffic.]
User: Show me an image of Porvoo with a car in motion.
Assistant: [Displays an image of Porvoo with a moving car retrieved via DigiTraffic.]
User: Plot them to compare

Other Formats