OAS->requests

OAS inspection

This implementation focuses on DigiTraffic with its OpenAPI specification file.

! [ -e "openapi.json" ] || wget https://tie.digitraffic.fi/swagger/openapi.json

Deep reference extraction

In order to generate tool schemas, we need to resolve and flatten the references to components.

Examples from DigiTraffic components:

refs = extract_refs(oas)
for i, (ref, obj) in enumerate(list(refs.items())[:1]):
    print(f"Reference: {ref}", list(obj.keys()))
    for j, (k,v) in enumerate(obj.items()):
        print(f"[{i}{j}]", k, json.dumps(v, indent=4) if k=='properties' else v)
Reference: #/components/schemas/WeatherSensorValueHistoryDto ['type', 'properties']
[00] type object
[01] properties {
    "roadStationId": {
        "type": "integer",
        "description": "Road station id",
        "format": "int64"
    },
    "sensorId": {
        "type": "integer",
        "description": "Sensor id",
        "format": "int64"
    },
    "sensorValue": {
        "type": "number",
        "description": "Sensor value",
        "format": "double"
    },
    "measured": {
        "type": "string",
        "format": "date-time",
        "writeOnly": true
    },
    "reliability": {
        "type": "string",
        "description": "Measurement reliability information",
        "enum": [
            "OK",
            "SUSPICIOUS",
            "FAULTY",
            "UNKNOWN"
        ]
    },
    "measuredTime": {
        "type": "string",
        "description": "Value's measured date time",
        "format": "date-time"
    }
}

source

extract_refs

 extract_refs (oas:dict)
Type Details
oas dict The OpenAPI schema
Returns dict The extracted references (flattened)

OAS schema to GPT-compatible schema

GPT currently recognizes only a limited number of descriptors when defining toolbox schema. Some of these descriptors (fields) can be directly transferred from OAS schema to toolbox, but many existing OAS schema fields will not be recognized by GPT and can cause errors. Therefore, transformation from OAS schemas to GPT-compatible schemas is necessary.

GPT currently recognizes these fields:

  1. type
    Specifies the data type of the value. Common types include:
  • string – A text string.
  • number – A numeric value (can be integer or floating point).
  • integer – A whole number.
  • boolean – A true/false value.
  • array – A list of items (you can define the type of items in the array as well).
  • object – A JSON object (with properties, which can be further defined with their own types).
  • null – A special type to represent a null or absent value.
  • any – Allows any type, typically used for flexible inputs or outputs.
  1. default: Provides a default value for the field if the user doesn’t supply one. It can be any valid type based on the expected schema.
  2. enum: Specifies a list of acceptable values for a field. It restricts the input to one of the predefined values in the array.
  3. properties: Used for objects, this defines the subfields of an object and their respective types.
  4. items: Defines the type of items in an array. For example, you can specify that an array contains only strings or integers.
  5. minLength, maxLength: Specifies minimum and maximum lengths for string parameters.
  6. minItems, maxItems: Specifies mininum and maximum number of items for array parameters.
  7. pattern: Specifies a regular expression that the string must match for string parameters.
  8. required: A list of required fields for an object. Specifies that certain fields within an object must be provided.
  9. additionalProperties: Specifies whether additional properties are allowed in an object. If set to false, no properties outside of those defined in properties will be accepted.

As such, we can extract corresponding fields from OAS schema, and converts all additional fields into parameter description.

source

transform_property

 transform_property (prop:dict, flatten_refs:dict={})
Type Default Details
prop dict The property to transform
flatten_refs dict {} The flattened references
Returns tuple The transformed property and whether it is a required property

Test usage with complex parameters from DigiTraffic endpoints:

parameters = [
    {
        "name": "lastUpdated",
        "in": "query",
        "description": "If parameter is given result will only contain update status.",
        "required": False,
            "schema": {
                "type": "boolean",
                "default": False
            }
    },
    {
        "name": "roadNumber",
        "in": "query",
        "description": "Road number",
        "required": False,
        "schema": {
            "type": "integer",
            "format": "int32"
        }
    },
    {
        "name": "xMin",
        "in": "query",
        "description": "Minimum x coordinate (longitude) Coordinates are in WGS84 format in decimal degrees. Values between 19.0 and 32.0.",
        "required": False,
        "schema": {
            "maximum": 32,
            "exclusiveMaximum": False,
            "minimum": 19,
            "exclusiveMinimum": False,
            "type": "number",
            "format": "double",
            "default": 19
        }
    },
    {
        "name": "yMin",
        "in": "query",
        "description": "Minimum y coordinate (latitude). Coordinates are in WGS84 format in decimal degrees. Values between 59.0 and 72.0.",
        "required": False,
        "schema": {
            "maximum": 72,
            "exclusiveMaximum": False,
            "minimum": 59,
            "exclusiveMinimum": False,
            "type": "number",
            "format": "double",
            "default": 59
        }
    },
    {
        "name": "xMax",
        "in": "query",
        "description": "Maximum x coordinate (longitude). Coordinates are in WGS84 format in decimal degrees. Values between 19.0 and 32.0.",
        "required": False,
        "schema": {
            "maximum": 32,
            "exclusiveMaximum": False,
            "minimum": 19,
            "exclusiveMinimum": False,
            "type": "number",
            "format": "double",
            "default": 32
        }
    },
    {
        "name": "yMax",
        "in": "query",
        "description": "Maximum y coordinate (latitude). Coordinates are in WGS84 format in decimal degrees. Values between 59.0 and 72.0.",
        "required": False,
        "schema": {
            "maximum": 72,
            "exclusiveMaximum": False,
            "minimum": 59,
            "exclusiveMinimum": False,
            "type": "number",
            "format": "double",
            "default": 72
        }
    }
]

for param in parameters:
    param, required = transform_property(param)
    print(json.dumps(param, indent=2))
    print(f"Required: {required}\n")
{
  "description": "If parameter is given result will only contain update status. (Name: lastUpdated; In: query)",
  "type": "boolean",
  "default": false
}
Required: False

{
  "description": "Road number (Name: roadNumber; In: query; Format: int32)",
  "type": "integer"
}
Required: False

{
  "description": "Minimum x coordinate (longitude) Coordinates are in WGS84 format in decimal degrees. Values between 19.0 and 32.0. (Name: xMin; In: query; Maximum: 32; Exclusivemaximum: False; Minimum: 19; Exclusiveminimum: False; Format: double)",
  "type": "number",
  "default": 19
}
Required: False

{
  "description": "Minimum y coordinate (latitude). Coordinates are in WGS84 format in decimal degrees. Values between 59.0 and 72.0. (Name: yMin; In: query; Maximum: 72; Exclusivemaximum: False; Minimum: 59; Exclusiveminimum: False; Format: double)",
  "type": "number",
  "default": 59
}
Required: False

{
  "description": "Maximum x coordinate (longitude). Coordinates are in WGS84 format in decimal degrees. Values between 19.0 and 32.0. (Name: xMax; In: query; Maximum: 32; Exclusivemaximum: False; Minimum: 19; Exclusiveminimum: False; Format: double)",
  "type": "number",
  "default": 32
}
Required: False

{
  "description": "Maximum y coordinate (latitude). Coordinates are in WGS84 format in decimal degrees. Values between 59.0 and 72.0. (Name: yMax; In: query; Maximum: 72; Exclusivemaximum: False; Minimum: 59; Exclusiveminimum: False; Format: double)",
  "type": "number",
  "default": 72
}
Required: False

Executing requests with GPT

Auxiliary function to generate requests

source

generate_request

 generate_request (function_name:str, url:str, method:str, path:dict={},
                   query:dict={}, body:dict={}, accepted_queries:list=[],
                   **kwargs)

Generate a request from the function name and parameters.

Type Default Details
function_name str The name of the function
url str The URL of the request
method str The method of the request
path dict {} The path parameters of the request
query dict {} The query parameters of the request
body dict {} The body of the request
accepted_queries list [] The accepted queries of the request
kwargs
Returns dict The response of the request 
generate_request(
    function_name="weathercamStations",
    url="https://tie.digitraffic.fi/api/weathercam/v1/stations/{id}",
    method="GET",
    path={"id": "C01504"}
)
{'type': 'Feature',
 'id': 'C01504',
 'geometry': {'type': 'Point', 'coordinates': [24.235601, 60.536727, 0.0]},
 'properties': {'id': 'C01504',
  'name': 'vt2_Karkkila_Korpi',
  'cameraType': 'HIKVISION',
  'nearestWeatherStationId': 1052,
  'collectionStatus': 'GATHERING',
  'state': None,
  'dataUpdatedTime': '2024-11-19T03:27:03Z',
  'collectionInterval': 600,
  'names': {'fi': 'Tie 2 Karkkila, Kappeli',
   'sv': 'Väg 2 Högfors, Kappeli',
   'en': 'Road 2 Karkkila, Kappeli'},
  'roadAddress': {'roadNumber': 2,
   'roadSection': 13,
   'distanceFromRoadSectionStart': 3818,
   'carriageway': 'ONE_CARRIAGEWAY',
   'side': 'LEFT',
   'contractArea': '',
   'contractAreaCode': 344},
  'liviId': 'Livi1089298',
  'country': None,
  'startTime': '1995-06-01T00:00:00Z',
  'repairMaintenanceTime': None,
  'annualMaintenanceTime': None,
  'purpose': 'keli',
  'municipality': 'Karkkila',
  'municipalityCode': 224,
  'province': 'Uusimaa',
  'provinceCode': 1,
  'presets': [{'id': 'C0150401',
    'presentationName': 'Poriin',
    'inCollection': True,
    'resolution': 'videoResolutionWidth=1280&videoResolutionHeight=720',
    'directionCode': '1',
    'imageUrl': 'https://weathercam.digitraffic.fi/C0150401.jpg',
    'direction': 'INCREASING_DIRECTION'},
   {'id': 'C0150402',
    'presentationName': 'Helsinkiin',
    'inCollection': True,
    'resolution': 'videoResolutionWidth=1280&videoResolutionHeight=720',
    'directionCode': '2',
    'imageUrl': 'https://weathercam.digitraffic.fi/C0150402.jpg',
    'direction': 'DECREASING_DIRECTION'},
   {'id': 'C0150409',
    'presentationName': 'Tienpinta',
    'inCollection': True,
    'resolution': 'videoResolutionWidth=1280&videoResolutionHeight=720',
    'directionCode': '9',
    'imageUrl': 'https://weathercam.digitraffic.fi/C0150409.jpg',
    'direction': 'SPECIAL_DIRECTION'}]}}

Toolbox schema

Extract important information about the functions and creates a GPT-compatible toolbox schema. The idea is to convert all necessary information for generating an API request to a parameter for GPT to provide. As such, the parameters of each function in this toolbox schema will include:

  • url: URL to send requests to (type string, with const default value formed with a base URL and endpoint path)
  • method: HTTP method for each endpoint (type string, with const value)
  • path: dictionary for path parameters that maps parameter names to schema
  • query: dictionary for query parameters that maps parameter names to schema
  • body: request body schema

Test with usage from DigiTraffic:

source

toolbox_schema

 toolbox_schema (base_url:str, oas:dict, service_name:Optional[str]=None,
                 fixup:Callable=None)

Form the toolbox schema from the OpenAPI schema.

Type Default Details
base_url str The base URL of the API
oas dict The OpenAPI schema
service_name Optional None The name of the service
fixup Callable None a fixup function to execute a REST API when a function name isn’t found.
Returns dict The toolbox schema 
tool_schema = toolbox_schema(base_url=BASE_URL, oas=oas, fixup=generate_request)
#pprint(tool_schema[:3])
for ts in tool_schema:
    if ts['function']['name']=='getLatestTrafficRestrictionNotificationById':
        print(json.dumps(ts['function'], indent=4))

GPT integration

Integrate with existing complete function:

from llmcam.fn_to_fc import complete, form_msgs, print_msgs
messages = form_msgs([
    ("system", "You are a helpful system administrator. Use the supplied tools to help the user."),
    ("user", "Get the weather camera information for the stations with ID C01503 and C01504."),
])
complete(messages, tool_schema)
print_msgs(messages)
>> System:
You are a helpful system administrator. Use the supplied tools to help the user.
>> User:
Get the weather camera information for the stations with ID C01503 and C01504.
>> Assistant:
Here is the weather camera information for the stations with ID C01503 and C01504:  ### Weather
Camera Station ID: C01503 - **Name:** kt51_Inkoo - **Camera Type:** BOSCH - **Location
Coordinates:** [23.99616, 60.05374] - **Nearest Weather Station ID:** 1013 - **Collection Status:**
GATHERING - **Data Updated Time:** 2024-11-19T03:25:41Z - **Collection Interval:** 600 seconds -
**Road Address:** Road 51, Section 14, Distance 422m - **Municipality:** Inkoo (Code: 149) -
**Province:** Uusimaa (Code: 1) - **Presets:**   - **Inkooseen:** \[Resolution: 1280x720\], [View
Image](https://weathercam.digitraffic.fi/C0150301.jpg)   - **Hankoon:** \[Resolution: 1280x720\],
[View Image](https://weathercam.digitraffic.fi/C0150302.jpg)   - **Tienpinta:** \[Resolution:
1280x720\], [View Image](https://weathercam.digitraffic.fi/C0150309.jpg)  ### Weather Camera Station
ID: C01504 - **Name:** vt2_Karkkila_Korpi - **Camera Type:** HIKVISION - **Location Coordinates:**
[24.235601, 60.536727] - **Nearest Weather Station ID:** 1052 - **Collection Status:** GATHERING -
**Data Updated Time:** 2024-11-19T03:27:03Z - **Collection Interval:** 600 seconds - **Road
Address:** Road 2, Section 13, Distance 3818m - **Municipality:** Karkkila (Code: 224) -
**Province:** Uusimaa (Code: 1) - **Presets:**   - **Poriin:** \[Resolution: 1280x720\], [View
Image](https://weathercam.digitraffic.fi/C0150401.jpg)   - **Helsinkiin:** \[Resolution: 1280x720\],
[View Image](https://weathercam.digitraffic.fi/C0150402.jpg)   - **Tienpinta:** \[Resolution:
1280x720\], [View Image](https://weathercam.digitraffic.fi/C0150409.jpg)   If you need more specific
details or other information, feel free to ask!