Replica DB API

The Replica DB API can be used to programmatically interact with the HeadSpin Replica DB. For a high-level overview of HeadSpin Performance Monitoring and User Flows, see Performance Monitoring UI. For more details about replica, see Replica DB.

Replica DB API Overview

HeadSpin provides some API routes to manage User Flows and their export into the Replica DB. In older versions of HeadSpin, User Flows were often referred to as <code class="dcode">perftests</code>. This terminology is being phased out but note that there might still be a few places that still use it.

Note also that there are settings and parameters that apply for the organization but can potentially be overridden (or otherwise have additional information) for individual User Flows; as such, some of the routes below may appear redundant.

The following legacy routes are deprecated:

List Organization Replica Settings

Fetch the current organization settings for replica. The settings that are configurable are described below:

These organization-level settings can be fetched with:

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/replica/settings
# Sample Output:
{
  "autosync": null,
  "table_settings": {
    "session_timeseries": {
      "export_timeseries_keys": [...]
    }
  }
}

Update Organization Replica Settings

Update the current organization settings for replica. The settings that are configurable are described here. The request should include the keys that should be updated. If a key for some option is omitted, that option will not be updated.

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/replica/settings -X PUT --data '{"autosync": true}'
# Sample Output:
{
  "status": "Settings updated successfully.",
  "status_code": 200
}

Fetch Replica DB Credentials

Retrieve the login credentials for the organization's Replica DB. This also includes the schema and tables that are shared across the organization.

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/replica/dbinfo
# Sample Output:
{
  "username": "headspin_iylb66",
  "host": "canary-replica-1.headspin.io",
  "password": "hg5DS9Oo7GTb",
  "db": "headspin_izlb66",
  "port": 5439,
  "connection_limit": 5,
  "tables": [
    {
      "table_type": "SESSION_TIMESERIES",
      "table_name": "session_timeseries",
      "columns": [
        ["session_id", "char(36)"],
        ["timeseries_key", "varchar(512)"],
        ["timeseries_name", "varchar(512)"],
        ["x_units", "varchar(512)"],
        ["y_units", "varchar(512)"]
      ]
    },
    ...
  ]
}

NOTE: The resulting schema might not include every table in replica, because some tables are specific to an individual User Flow. See here to fetch the full schema pertaining to an individual User Flow.

Listing User Flows

Retrieve the list of user flows for the organization:

curl https://<your_api_token>@api-dev.headspin.io/v0/userflows
# Output:
{
  "user_flows": [
    {
      "created_time": 1573863722.48,
      "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
      "description": null,
      "name": "Random User Flow"
    },
    ...

Alternatively, the information for a single User Flow can be returned with:

curl https://<your_api_token>@api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba
# Output:
{
  "created_time": 1574373698.96,
  "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
  "description": "Some test user flow."
  "name": "Random User Flow"
}

Get Session Status in User Flow

Retrieve the status of a session in its User Flow:

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/userflows/session/{session_id}
# Output
{
  "status": "Passed",
  "perf_task_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
  "user_flow_name": "Random User Flow",
  "event_info": {
    "status": "Passed",
    "host": "proxy-us-mvo-2.headspin.io",
    "status_message": null,
    "event_time": 1620059726,
    "device_id": "00008020-0001546E21E1002E"
  },
  "session_id": "90770483-ac2d-11eb-9845-06c4c57ba6cd",
  "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
  "perf_event_id": "fcdb6dc1-ac2d-11eb-9851-06c4c57ba6cd",
  "status_message": null
}

If the session is not attached to a User Flow, this will return a 404 status code:

{
  "status": "No User Flow info for session ID: 30e002b5-a91f-11eb-bf66-06c4c57ba6cd",
  "status_code": 404
}

Add Session to User Flow

Add the session to the requested User Flow, with the given status (and optional status message):

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/userflows/session/{session_id} -X POST --data '{"user_flow_id": "", "status": "PASSED", "status_message": "My message"}'
# Output
{
  "status_code": 200,
  "status": "Successfully added session to User Flow."
}

NOTE: The request will fail if the session is already assigned to another User Flow.

Update Session in User Flow

Update the session in its current User Flow with the new status (and optional status message):

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/userflows/session/{session_id} -X PUT --data '{"status": "FAILED", "status_message": "It failed, actually."}'
# Output
{
  "status_code": 200,
  "status_message": "Successfully updated the status."
}

The session must already be assigned to a User Flow for this request.

Remove Session from User Flow

Remove the session from its current User Flow:

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/userflows/session/{session_id} -X DELETE
# Output
{
  "status_code": 200,
  "status_message": "Session successfully removed from its current User Flow."
}

This does not delete the session, but simply removes it from the current User Flow. In order to move a session from one User Flow to another, simply remove the session from its current one first, then re-add it to the new User Flow.

Check Export Status for Session

Check for the export status of a given session:

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/userflows/session/{session_id}/export/status
# Output
{
  "recent_export": {
    "status": "success",
    "last_export_timestamp": 1632844515.803,
    "export_id": "08a6590d-7c2e-40a8-9ee4-27963bf41129",
    "session_id": "90770483-ac2d-11eb-9845-06c4c57ba6cd",
    "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
    "perf_event_id": "fcdb6dc1-ac2d-11eb-9851-06c4c57ba6cd",
  }
}

The response will include the <code class="dcode">session_id</code>, <code class="dcode">perf_event_id</code>, <code class="dcode">export_id</code> (an ID that can be used with the route here), and the timestamp that the export was completed for this session (in seconds since UNIX epoch). It will also include a status field, which can be one of:

• <code class="dcode">not_done</code>: Implies the session is scheduled for a sync, which has not yet completed.
• <code class="dcode">success</code>: Implies the session has successfully synced.
• <code class="dcode">dirty</code>: Implies the session has synced, but it might not have exported with the latest data. Syncing this User Flow will likely schedule this session to sync again.
• <code class="dcode">error</code>: Implies the session synced, but there was an error.
• <code class="dcode">cancelled</code>: Implies this session's sync was cancelled.

NOTE: These statuses overlap with the statuses here.

If the session is not a part of any User Flow, the response will simply be:

{"recent_export": null}

and none of the fields above will be in the response.

Check Recent Exports for User Flow

Check for any recently completed exports (within the past 24 hours) for the given User Flow:

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/userflows/{user_flow_id}/exports/recent
# Output
{
  "recent_exports": [
    {
      "status": "success",
      "last_export_timestamp": 1632844515.803,
      "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
      "perf_event_id": "3a3e22aa-0817-11ea-a4d3-00e04c6914ce",
      "export_id": "f95cad7b-8977-4e99-883e-0359a7b86c40",
      "session_id": "38c56e38-0817-11ea-a4d3-00e04c6914ce"
    },
    # ...
  ]
}

If <code class="dcode">exported_after</code> is passed (in seconds since UNIX epoch), only exports completed after this time will be returned. When this parameter is not passed, a default value of one day before the current time is assumed. An example that passes an explicit timestamp for <code class="dcode">exported_after</code> is shown here:

curl -H "Authorization: Bearer <your_api_token>" 'https://api-dev.headspin.io/v0/userflows/{user_flow_id}/exports/recent?exported_after=1632844515'
# Output
{
  "recent_exports": [
    {
      "status": "success",
      "last_export_timestamp": 1632844515.803,
      "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
      "perf_event_id": "3a3e22aa-0817-11ea-a4d3-00e04c6914ce",
      "export_id": "f95cad7b-8977-4e99-883e-0359a7b86c40",
      "session_id": "38c56e38-0817-11ea-a4d3-00e04c6914ce"
    },
    # ...
  ]
}

Exported Session Statistics for the Organization

Return the number of exported sessions (grouped by User Flow ID) for the organization.

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/replica/export/status
# Output
{
  "1c89aa46-0807-11ea-bee9-02db88d209ba": {
    "exported": 2,
    "total": 4
  }
}

NOTE: This may return User Flows that were never exported as well.

List DB Credentials and User Flow Schema

Retrieve the login credentials for the organization's Replica DB:

curl https://<your_api_token>@api-dev.headspin.io/v0/userflows/dbinfo
# Output
{
  "username": "headspin_iylb66",
  "host": "canary-replica-1.headspin.io",
  "password": "hg5DS9Oo7GTb",
  "db": "headspin_izlb66",
  "port": 5439,
  "connection_limit": 5,
  "tables": [
    {
      "table_type": "SESSION_TIMESERIES",
      "table_name": "session_timeseries",
      "columns": [
        ["session_id", "char(36)"],
        ["timeseries_key", "varchar(512)"],
        ["timeseries_name", "varchar(512)"],
        ["x_units", "varchar(512)"],
        ["y_units", "varchar(512)"]
      ]
    },
    ...
  ]
}

Note: There is a limit to the number of simultaneous connections that can be opened with these credentials (default: 5). Using Grafana will also use connections that count against this limit. If more simultaneous connections are needed, contact HeadSpin support.

A similar command can be run to fetch the same information, but also include additional tables that may only pertain to this user flow. This command is:

curl https://<your_api_token>@api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/dbinfo
# Output
{
  "username": "headspin_iylb66",
  "host": "canary-replica-1.headspin.io",
  "password": "hg5DS9Oo7GTb",
  "db": "headspin_izlb66",
  "port": 5439,
  "connection_limit": 5,
  "tables": [
    {
      "table_type": "PERF_TASK",
      "table_name": "random_user_flow",
      "columns": [ ... ]
    },
    ...
  ]
}

The <code class="dcode">table_type</code> field can be used as a stable type name, instead of relying on the table name which may change. Some of these table types and the replica schema in general are outlined in more detail here.

Trigger Replica DB Sync

User Flows need to be synced into the Replica DB before the Replica DB can be queried. To request a sync, run:

curl -X POST 'https://<your_api_token>@api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/dbsync'
# Possible Output:
{
  "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
  "status": "not_done",
  "export_id": "08a6590d-7c2e-40a8-9ee4-27963bf41129"
}

This will request that the User Flow is synced. If no parameters are passed, then the request should return an export_id field that can be used to check for the status of the sync later.

It is also possible to pass in a custom <code class="dcode">timeout</code> parameter like below:

curl -X POST 'https://<your_api_token>@api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/dbsync?timeout=100'
# Possible Output:
{
  "_headspin_progress": "..",
  "status": "success",
  "export_id": "bd361ce7d24c4861b22716fe127ce0c4"
}

This <code class="dcode">timeout</code> parameter (when explicitly passed) is interpreted the same as for listening on an existing sync.

Replica DB Sync for Specific Sessions

It is also possible to trigger a sync for specific sessions within a user flow.

curl -X POST -H "Authorization: Bearer <your_api_token>" 'https://api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/sessions/dbsync'  \
     --data '{"session_ids": <list of session_ids>}'
# Possible Output:
{
  "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
  "status": "not_done",
  "export_id": "08a6590d-7c2e-40a8-9ee4-27963bf41129"
}

If for some reason sessions in a user flow are not syncing as expected, include <code class="dcode">force: true</code> in the request payload to force scheduling this user flow to sync the requested sessions. This will schedule the sync regardless of autosync settings.

curl -X POST -H "Authorization: Bearer <your_api_token>" 'https://api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/sessions/dbsync'  \
     --data '{"session_ids": <list of session_ids>, "force": true}'

Similar to <code class="dcode">/v0/userflows/dbsync</code>, this route also accepts a timeout parameter to optionally wait on the export to be fully scheduled. This can be specified as a query argument on the URL or within the request payload timeout: <code class="dcode">&lt;int></code>.

# The following are equivalent:
curl -X POST -H "Authorization: Bearer <your_api_token>" 'https://api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/sessions/dbsync?timeout=20'  \
     --data '{"session_ids": <list of session_ids>, "force": true}'
curl -X POST -H "Authorization: Bearer <your_api_token>" 'https://api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/sessions/dbsync'  \
     --data '{"session_ids": <list of session_ids>, "force": true, timeout: 20}'

Listen on an Existing Sync

When given an <code class="dcode">export_id</code>, it is possible to check on the status of the resulting sync by running the following:

curl https://<your_api_token>@api-dev.headspin.io/v0/userflows/export/bd361ce7d24c4861b22716fe127ce0c4
# Possible output.
{
  "_headspin_progress": "..",
  "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba",
  "status": "success",
  "export_id": "bd361ce7-d24c-4861-b227-16fe127ce0c4"
}

This command respects a <code class="dcode">timeout</code> parameter to optionally wait for the User Flow to sync up to the given <code class="dcode">export_id</code>. If no <code class="dcode">timeout</code> parameter is passed, it is assumed to be negative. The possible values for timeout are listed below:

• If <code class="dcode">timeout > 0</code>: wait for up to <code class="dcode">timeout</code> seconds for the sync to complete before returning.
• If <code class="dcode">timeout < 0</code>: wait indefinitely for the sync to complete.
• If <code class="dcode">timeout = 0</code>: return with the details of the export and exit.

This works by sending a 200 status code, then sending chunked JSON data back to the client. When invoking this API in this manner, the status field in the JSON response will have a more descriptive response as below:

• <code class="dcode">success</code>: The replica sync (up to the time of the given export_id) finished successfully.
• <code class="dcode">error</code>: There was an error in the replica sync.
• <code class="dcode">timeout</code>: The request timed out before the replica sync was completed.
• <code class="dcode">cancelled</code>: The replica sync request was cancelled.

If the request times out or otherwise drops, the same request can be made again.

Note: When waiting on the sync, it is possible for the connection to drop in some cases, even with the server sending chunked JSON data every so often to keep the connection alive, if the <code class="dcode">timeout</code> parameter is set for too long. For example, some corporate proxies automatically close connections open for longer than a few minutes. Keep this in mind when setting the <code class="dcode">timeout</code>.

Remove User Flow From Replica

To remove a Replica DB Sync on User Flow with ID <code class="dcode">1c89aa46-0807-11ea-bee9-02db88d209ba</code>, we use the same route as above but using the HTTP <code class="dcode">DELETE</code> method instead of <code class="dcode">POST</code>. This route does not delete the User Flow, but instead removes the User Flow from the Replica DB:

curl -X DELETE https://<your_api_token>@api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/dbsync
# Output:
{"status": "OK", "status_code": 200}

Warning: The delete command deletes the user flow export from the Postgres Replica DB. After a delete, the state of the system is as if the user flow was never exported to the Replica DB.

View User Flow Replica Settings (Autosync)

Each User Flow has a few settings defined for replica, most notably autosync. In order to view these settings for the User Flow with ID <code class="dcode">1c89aa46-0807-11ea-bee9-02db88d209ba</code>, run:

curl -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/settings
# Output:
{
  "autosync": false,
  "user_flow_id": "1c89aa46-0807-11ea-bee9-02db88d209ba"
}

NOTE: The results will return the settings (including any overrides) for this User Flow; if this User Flow does not have any explicit override set, it will fallback to the value used for the organization settings (as returned from here).

Update User Flow Replica Settings (Autosync)

To update the settings for a user flow, run the same route as above, but using a POST instead of a GET, and passing in a JSON object with the settings to update:

curl -X POST -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/userflows/1c89aa46-0807-11ea-bee9-02db88d209ba/settings --data '{"autosync": true}'
# Output:
{"status": "Updated settings successfully.", "status_code": 200}

NOTE: This overrides the organization settings for this specific User Flow. It does not change this setting for other User Flows. Passing <code class="dcode">null</code> for a setting should revert this User Flow to use whatever is configured for the organization (as returned from here).

Legacy Replica Routes

These routes are also still supported, but are deprecated and may eventually be removed in the future. The returned field <code class="dcode">perf_test_id</code> is equivalent to the <code class="dcode">user_flow_id</code> field in the rest of the system. These routes are analogous to their more modern <code class="dcode">/userflows/</code> equivalents.

Global DB Credentials

Legacy command to retrieve the login credentials for the organization's Replica DB:

curl https://<your_api_token>@api-dev.headspin.io/v0/perftests/dbinfo
# Output:
{"username": "headspin_6pulfy", "host": "localhost", "password": "XT7qGHtu11Ra", "db": "headspin_6pulfy", "port": 5432}

Note: There is a limit to the number of simultaneous connections that can be opened with these credentials. Bear this in mind for different consumers for replica. If more simultaneous connections are needed (default is 5), contact HeadSpin to increase the limit.

User Flow DB Credentials

The legacy route to retrieve login credentials and pertinent database schema for an exported user flow with ID <code class="dcode">4bd7cdee-240c-11e9-b8cd-024337074d38</code> is still supported:

curl https://<your_api_token>@api-dev.headspin.io/v0/perftests/4bd7cdee-240c-11e9-b8cd-024337074d38/dbinfo
# Output:
{
  "host": "localhost",
  "db": "headspin_6pulfy",
  "username": "headspin_6pulfy",
  "password": "XT7qGHtu11Ra",
  "port": 5432
  "perftest_id": "4bd7cdee-240c-11e9-b8cd-024337074d38",
  "tables": [
    {
      "table_type": "PERF_TASK",
      "table_name": "random_user_flow",
      "columns": [...]
    },
    {
      "table_type": "SESSIONS",
      "table_name": "capture_sessions",
      "columns": [...]
    },
    {
      "table_type": "SESSION_TIMESERIES_DATA",
      "table_name": "session_timeseries_data",
      "columns": [...]
    },
    {
      "table_type": "ISSUES",
      "table_name": "capture_issues",
      "columns": [...]
    },
    {
      "table_type": "SESSION_TIMESERIES",
      "table_name": "session_timeseries",
      "columns": [...]
    }
  ]
}

The <code class="dcode">table_type</code> field can be used as a stable type name, instead of relying on the table name which may change. Some of these table types are outlined in more detail here

Trigger a Replica DB Sync

To trigger a Replica DB Sync on user flow with ID <code class="dcode">4bd7cdee-240c-11e9-b8cd-024337074d38</code>, run:

curl -X POST https://<your_api_token>@api-dev.headspin.io/v0/perftests/4bd7cdee-240c-11e9-b8cd-024337074d38/dbsync
# Output:
{"status": "OK", "status_code": 200}

Delete a Replica DB Sync

To delete a Replica DB Sync on user flow with ID <code class="dcode">4bd7cdee-240c-11e9-b8cd-024337074d38</code>, we use the same route as above but using the HTTP <code class="dcode">DELETE</code> method instead of <code class="dcode">POST</code>.

curl -X DELETE https://<your_api_token>@api-dev.headspin.io/v0/perftests/4bd7cdee-240c-11e9-b8cd-024337074d38/dbsync
# Output:
{"status": "OK", "status_code": 200}

Warning: The delete command deletes the user flow export from the Postgres Replica DB. After a delete, the state of the system is as if the user flow was never exported to the Replica DB.