HeadSpin Documentation
Documentation

Session API

Session Capture API

Get all sessions in your org

Fetch timestamps for a session

Create a session

Stop a session

Get the number of TLS exceptions for each host in a session

Get all sessions in your org

Route Method
/v0/sessions?include_all={bool}&num_sessions={int}&tag={string}:{string} GET

Optional Parameters

  • <code class="dcode">?include_all={true|false}</code>: If <code class="dcode">true</code>, this will return both live (active) sessions and ended (inactive sessions). If this value is <code class="dcode">false</code>, only live sessions will be included. The default value is <code class="dcode">false</code>.
  • <code class="dcode">?num_sessions={n}</code>: The number of sessions you want the API request to return, sorted by most recent. The default value is <code class="dcode">10</code>.
  • <code class="dcode">?tag={key}:{value}</code>: Select sessions that contain the tag defined by tag key <code class="dcode">key</code> and tag value <code class="dcode">value</code>. Spaces in either the tag key or value may be represented using <code class="dcode">+</code>. For example, sessions containing the tag <code class="dcode">my example: this is an example tag</code> could be selected using the query argument <code class="dcode">?tag=my+example:this+is+an+example+tag</code>. This query argument may be provided multiple times to select sessions containing at least one of the tags provided.
  • <code class="dcode">?next_token={value}</code>: Use this value as a start index to continue from a previous request.

Example

To get a list of the most recent 20 sessions, both active and inactive:


curl 'https://<your_api_token>@api-dev.headspin.io/v0/sessions?include_all=true&num_sessions=20'

To get sessions containing at least one of the tags <code class="dcode">my:tag</code> or <code class="dcode">my other:example tag</code>:


curl 'https://<your_api_token>@api-dev.headspin.io/v0/sessions?include_all=true&num_sessions=20&tag=my:tag&tag=my+other:example+tag'

Response

The response is a JSON object with the <code class="dcode">sessions</code> key, and it contains a list of sessions in your org matching the optional parameters <code class="dcode">include_all</code> and <code class="dcode">num_sessions</code>. For pagination, a <code class="dcode">next_token</code> value is also provided from which to continue in the next request.


{
    "sessions": [
        {
            "companion_id": null,
            "session_type": "capture",
            "start_time": 1689121253.742,
            "state": "ended",
            "session_id": "<session_id>",
            "device_id": "<device_id>"
        },
        {
          "companion_id": null,
          "session_type": "capture",
          "start_time": 1689121238.493,
          "state": "ended",
          "session_id": "<session_id>",
          "device_id": "<device_id>"
        }
    ],
    "next_token": "1316074"
}

NOTE: The last page may contain up to <code class="dcode">num_sessions</code> sessions and their <code class="dcode">next_token</code>. A response of <code class="dcode">{"sessions": [], "next_token": null}</code> indicates the end of the set matching the given query. The type of <code class="dcode">next_token</code> should be assumed to be a <code class="dcode">string</code> or <code class="dcode">null</code>.

Fetch timestamps for a session

Route Method
/v0/sessions/{session_id}/timestamps GET

Fetch all timestamps associated with capture for the target session.

Example


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}/timestamps

Response

If the request is successful, a <code class="dcode">HTTP 200 OK</code> response is returned with a JSON object containing key-value pairs for any available timestamps associated with the capture session ID.


{
    "capture-complete": 1579113687.702,
    "capture-ended": 1579113686.105,
    "capture-started": 1579113669.27
}

A <code class="dcode">HTTP 404</code> response is returned if no timestamps are available or the session does not exist.

Create a session

Route Method
>/v0/sessions POST

To start a session on a device, you must first lock that device. Lock the device through the HeadSpin UI or using the Android Device API or iOS Device API. Browsers require starting and stopping a capture session using the HeadSpin Selenium capture capabilities. Please refer to Selenium Capabilities and Chrome DevTools.

Request Body

The request body must contain a JSON object with a minimum of:


{
    "session_type": "capture",
    "device_address": "<device_address>"
}

Optional session configuration parameters:

Key Name Description
allow_replace If the target device is currently locked by the same user, and has an active capture session, then setting allow_replace to true will stop the current session and start a new one. If allow_replace is false, then the request will fail in that circumstance. Default true.
capture_log A boolean value that specifies whether to capture device log. Default true.
capture_sys A boolean value that specifies whether to capture certain system information, for example battery drain. Default true.
capture_function_call A boolean value that specifies whether to capture function call data. Default true.
capture_network A boolean value that specifies whether to capture network data. Default true.
capture_video A boolean value that specifies whether to capture a video of the device screen. Default true.
device_id The ID of the device to start the session on.
disable_http2 A boolean value that specifies whether to prevent the device from using HTTP/2. Default false.
disable_server_cert_check A boolean value that specifies whether to verify server TLS certificates during network capture. Default false.
mitm_ignore A JSON array of hosts to ignore. Default [] has no effect. See Actions for Hosts with TLS Exceptions for more details.
network_keep_in_session_patterns An optional JSON array containing patterns for hosts to keep in the network capture. Default [] has no effect. Has the opposite effect of mitm_ignore. See Actions for Hosts with TLS Exceptions for more details.
network_shaping This optional JSON object specifies network conditioning targets. For example, "network_shaping": {"up": 1.2, "down": 5.1, "rtt": 10, "loss": 0.1}. The settings "up" and "down" apply limits on the upload and download rates in units of megabits per second (Mbps); "rtt" simulates adding an additional round-trip time delay in units of seconds; "loss" simulates a further packet loss rate as a fraction, expressed as a number between 0 and 1 (received packets fraction = (1 − native link loss) * (1 − simulated loss)). Only the settings that are specified are conditioned.
nowait A boolean value indicating if the request should return without waiting for the session to be ready. If the value is false, the request will return a response only after the session is ready to capture data. If the value is true, the request will initiate starting the session but will return immediately, without waiting for the device to be ready to capture data. Then, you need to check whether the capture session is ready by polling Fetch timestamps for a session. Once the timestamps endpoint returns HTTP 200 OK and capture-started, the session is ready to capture data. Default false.

Example


curl -X POST https://<your_api_token>@api-dev.headspin.io/v0/sessions -d '{"session_type": "capture","device_address": "{android-device-serial}@{hostname}"}'

Response

If the request is successful, the response is a JSON object with details of the created session, including <code class="dcode">session_id</code>.


{
    "companion_id": null,
    "session_type": "capture",
    "start_time": 1571175936.264,
    "endpoints": [],
    "state": "active",
    "session_id": "<session_id>",
    "device_id": "<device_id>"
}

If the request fails, you may get:

  • <code class="dcode">{"status": "Device is currently in use by someone else.", "status_code": 403}</code>: Someone else is currently using the device. Choose another device to create a session, or wait until this device is free.
  • <code class="dcode">{"status": "Invalid device ID.", "status_code": 404}</code>: The device ID specified doesn't exist. Double check that the device ID is correct.
  • <code class="dcode">{"status": "Device must be locked before capture.", "status_code": 403}</code>: The device must be locked by you before a session can be created. Lock the device through the HeadSpin UI or an API call.

The <code class="dcode">nowait</code> option may help if you run Espresso tests on a device and the Espresso test code itself calls the session start endpoint. The default <code class="dcode">nowait: false</code> value keeps the HTTP connection until the session starts; however, as network capture setup happens the device will temporarily disconnect from the network causing HTTP connections to disconnect, resulting in the start session request returning an error.

To avoid such a situation, you can use the <code class="dcode">nowait: true</code> option, which starts a capture session and immediately responds without waiting for the session to reach the ready state. Then, you can determine if the session is ready for testing by polling the <code class="dcode">timestamps</code> endpoint until requests return <code class="dcode">HTTP 200 OK</code>, indicating that the capture session is ready for testing. Note that errors may occur while polling the <code class="dcode">timestamps</code> endpoint since the device will temporarily disconnect from the network as it prepares for network capture. Such errors should be ignored and polling should continue until the <code class="dcode">HTTP 200 OK</code> response is received.

Actions for Hosts with TLS Exceptions

When using hosts with pinned TLS certificates, one of these two actions should be taken to enable accurate and comprehensive network capture:

If it is not possible to trust the HeadSpin cert or enable profiling via instrumentation, hosts with pinned TLS certificates may fail to establish secure connections with our network capture setup, which can cause TLS Exceptions and undesirable app performance degradations. To prevent this from affecting your tests, hosts experiencing TLS exceptions may be filtered from the network capture using the <code class="dcode">mitm_ignore</code> argument.

The <code class="dcode">mitm_ignore</code> argument accepts an array of ignored <code class="dcode">host:port</code> or <code class="dcode">ip:port</code> regex patterns. If just the host name is provided, all ports will be matched. These hosts are ignored from the network capture session.

Example usage may include the following: <code class="dcode">['abc\.example\.com:443', '.*\.mydomain.com']</code>

The <code class="dcode">network_keep_in_session_patterns</code> argument controls whether a destination <code class="dcode">host:port</code> (or <code class="dcode">ip:port</code>) is kept in the session after an error. Add glob patterns here to keep a destination with an error in the session, to continue to try to parse the network protocol for that destination.

Stop a session

Route Method
/v0/sessions/{session_id} PATCH

Browsers require starting and stopping a capture session using the HeadSpin Selenium capture capabilities. Please refer to Selenium Capabilities and Chrome DevTools.

Request Body

The request body must be this JSON object:


{ "active": false }

Optional session configuration parameters:

Key Name Description
active A boolean value that makes the session stop.
nowait A boolean value indicating if the request should return without waiting for the session to end. If the value is false, the request will return a response only after the session has ended completely. If the value is true, the request will initiate ending the session but will return immediately, without waiting for the session to end completely. Fetch timestamps for a session returns HTTP 200 OK with the capture-complete key and value when the session ended completely. Default false.

Example


curl -X PATCH https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id} -d '{"active": false}'

Response

  • A <code class="dcode">HTTP 200 OK</code> response with <code class="dcode">{"msg": "Video uploaded to https://api-dev.headspin.io/v0/sessions/{session_id}.mp4"}</code> is returned. This message contains a link to the captured video. The name of the video is the session ID.
  • A <code class="dcode">HTTP 500</code> response if the request fails. Check that you have the right JSON object in the request body.
  • A <code class="dcode">HTTP 404</code> Not Found response if the request fails. Make sure that the session id in your request is valid.

The <code class="dcode">nowait</code> option may help if you run Espresso tests on a device and the Espresso test code calls the session stop endpoint. The default <code class="dcode">nowait: false</code> value will keep the HTTP connection until the session has completely ended. However, as the session ends and network capture terminates, the device disconnects from the network temporarily causing the request to fail with a network disconnect error. To avoid such a situation, you can use <code class="dcode">nowait: true</code> value to stop a capture session, but return a response immediately without waiting for the session to end completely. In this case, you can determine if the session has ended completely by polling the <code class="dcode">timestamps</code> endpoint until it returns a <code class="dcode">HTTP 200 OK</code> response and the body contains the <code class="dcode">capture-complete</code> key indicating that the capture session has ended completely. Note that errors may occurs while polling the <code class="dcode">timestamps</code> endpoint since the device will temporarily disconnect from the network as it terminates network capture. In this case, the error should be ignored and polling should continue until an <code class="dcode">HTTP 200 OK</code> response is received.

Delete a session

Route Method
/v0/sessions/{session_id} DELETE

Permanently delete a session and all associated data. Use with caution!

Example


curl -X DELETE https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}

Response

  • A <code class="dcode">HTTP 200 OK</code> response if the session was successfully deleted.
  • A <code class="dcode">HTTP 409 Conflict</code> response if the session is still active and as a result cannot yet be deleted.
  • A <code class="dcode">HTTP 500</code> response if the request fails. Make sure that the session id in your request is valid.

Get the number of TLS exceptions for each host in a session

Route Method
/v0/sessions/{session_id}/tlsexceptions GET

Example


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}/tlsexceptions

Response

The response is a JSON object where each key is a host name, and the value is the number of TLS exceptions for that host.


{
    "abc.example.com:443": 4,
    "www.mydomain.com:443": 2,
}

If the request contains an invalid session ID (such as one that doesn't exist), a <code class="dcode">404</code> response will be returned.

Import a Video Analysis Session

Route Method
/v0/sessions/import/video POST

External videos in a MP4 container may be imported to create a HeadSpin video analysis session.

Example

To upload a video using <code class="dcode">curl</code>, use the <code class="dcode">--data-binary "@<path_to_your_mp4>"</code> syntax, for example <code class="dcode">--data-binary "@/Users/headspin/somevideo.mp4"</code>.


curl -X POST https://<your_api_token>@api-dev.headspin.io/v0/sessions/import/video --data-binary "@/path/to/yourvideo.mp4"

Response


{
    "session_id": "4cbb37f0-8955-4ba4-a897-31759e0062e5",
    "url": "https://ui.headspin.io/sessions/4cbb37f0-8955-4ba4-a897-31759e0062e5/waterfall"
}

Download Data for a Session

Download captured session data

Download a summary of session Issues

Retrieve the most significant Issue by Issue Burst

Get the session video metadata

Download captured session data

Route Method
/v0/sessions/{session_id}.{ext} GET

{ext} (extension) specifies what type of data to download for this session. Extension must be one of:

  • <code class="dcode">mp4</code>: HeadSpin screen recording for this session. This extension supports a <code class="dcode">fps</code> query parameter with a positive integer value greater than 1 and less than 120. If the <code class="dcode">fps</code> parameters is provided, the screen recording video will be resampled to a constant frame rate at the specified frame rate.
  • <code class="dcode">har</code>: Standardized HTTP message archive, containing all HTTP traffic content as a JSON object. This extension supports an <code class="dcode">enhanced</code> query parameter that (if <code class="dcode">True</code>) will enhance the HTTP message archive with the actual HTTP body content extracted from the binary network traffic capture. Entries can be temporally filtered by supplying a <code class="dcode">label_id</code> query parameter corresponding to a session label that spans the desired temporal range. Currently only supports HTTP/1.X.
  • <code class="dcode">mar</code>: HeadSpin's variant of HTTP message archive, containing additional metadata. This is also a JSON object.
  • <code class="dcode">csv</code>: <code class="dcode">mar</code> data in <code class="dcode">.csv</code> form
  • <code class="dcode">pcap</code>: Binary network traffic capture. See Reading Network Packet Data.
  • <code class="dcode">sslkeylog.txt</code>: TLS/SSL master secret key log for the pcap data. See Reading Network Packet Data.
  • <code class="dcode">appium.log.gz</code> or <code class="dcode">appium.log</code>: Appium session log. The log is available for all sessions, including capture sessions and non-capture sessions.
  • <code class="dcode">selenium.log.gz</code> or <code class="dcode">selenium.log</code>: Selenium driver session log. The log is available for all sessions, including capture sessions and non-capture sessions.
  • <code class="dcode">device.log.gz</code> or <code class="dcode">device.log</code>: Device event log.
  • <code class="dcode">jsconsole.log.gz</code> or <code class="dcode">jsconsole.log:</code> JavaScript console log. <code class="dcode">headspin:enableJSConsoleLog</code> capability collects them. Please read Appium Capabilities and Selenium Capabilities for more details about the capability. The log is available for all sessions, including capture sessions and non-capture sessions.

Examples

To save the downloaded data as files on your computer, use the flag -o {filepath} in your cURL commands.

To download the video screen recording:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.mp4 -o 'video.mp4'

To download the video screen recording resampled to a constant frame rate of 15 frames per second:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.mp4\?fps\=15 -o 'video_15fps.mp4'

To download the enhanced HAR data containing request and response bodies for HTTP/1.X traffic:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.har?enhanced=True -o 'data.har'

To download the <code class="dcode">pcap</code> data:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.pcap -o 'data.pcap'

To download the TLS/SSL master secret:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.sslkeylog.txt -o 'sslkeylog.txt'

To download the Appium log:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.appium.log.gz -o 'appium.log.gz'

To download the Selenium driver log:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.selenium.log.gz -o 'selenium.log.gz'

To download the device log:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.device.log.gz -o 'device.log.gz'

To download the device log:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}.device.log.gz -o 'device.log.gz'

Response

The response is a stream of the content of the data file. To save the data into a file on your disk, use the <code class="dcode">-o {filepath}</code> flag in your <code class="dcode">cURL</code> request.

Reading Network Packet Data

The downloaded packet capture (PCAP) is binary data, with the network traffic encrypted using TLS/SSL. To read this data, you must use a tool that can read <code class="dcode">pcap</code> packets, and decrypt the traffic data with the <code class="dcode">sslkeylog.txt</code> file. To do this:

  1. Download Wireshark.
  2. Open Wireshark. In Wireshark, open the pcap file with <code class="dcode">File > Open</code>, and select the <code class="dcode">pcap</code> file.
  3. Next, you'll need to speciy the key log file. Go to <code class="dcode">Wireshark > Preferences > Protocols > TLS</code>.
  4. In <code class="dcode">(Pre)-Master-Secret log filename</code>, select the downloaded <code class="dcode">sslkeylog.txt</code> file.
  5. The traffic data should now be decrypted. You can read the <code class="dcode">pcap</code> data in Wireshark.

Alternatively, you can open and decrypt a pcap directly using the <code class="dcode">wireshark</code> command line interface with wireshark <code class="dcode">-o ssl.keylog_file:path/to/sslkeylog.txt path/to/file.pcap,/code>.

Download a summary of session Issues

This data is the same as that shown in the Waterfall UI issue card.

Route Method
/v0/sessions/analysis/issues/{session_id} GET

Optional Parameters

  • <code class="dcode">orient</code>: This parameter can have the value <code class="dcode">column</code> (default) or <code class="dcode">record</code>. This specifies the return format of the data:
  • The <code class="dcode">column</code> orientation outputs the data by Issue category and then by column of data available for that issue (e.g. Status, Impact Time, Content Type...).
  • The <code class="dcode">record</code> orientation outputs the data by Issue category and then by each instance of the issue.

Example


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/analysis/issues/ed8ba23d-1092-11ea-a8da-acde48001122

To get the data by issue category and by issue:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/analysis/issues/ed8ba23d-1092-11ea-a8da-acde48001122?orient=record

Response

Example of a column response:


{
    "Low Frame Rate": {
        "Impact Time (sec)": [
            "0.39"
        ],
        "Issue Start": [
            "00:00:22.225"
        ],
        "Total Low FPS Time (sec)": [
            "1.39"
        ],
        "Impact/Total (%)": [
            "28.3"
        ]
    },
    "Domain Sharding": {
        "Domain": [
            "example.com",
            "example2.com",
            "unity3d.com"
        ],
        "Total Connection Time (ms)": [
            "675",
            "170"
        ],
        "Impact Time (ms)": [
            "301",
            "55"
        ],
        "Impacted Connection Count": [
            "1",
            "1"
        ],
        "Subdomain Count": [
            "2",
            "2"
        ],
        "Impact/Total (%)": [
            "44.6",
            "32.4"
        ]
    }
}

Example of a record response:


{
    "Low Frame Rate": [
        {
            "Impact Time (sec)": 0.39,
            "Issue Start": "00:00:22.225",
            "Total Low FPS Time (sec)": 1.39,
            "Impact/Total (%)": 28.3
        }
    ],
    "Domain Sharding": [
        {
            "Domain": "example.com",
            "Total Connection Time (ms)": 675,
            "Impact Time (ms)": 301,
            "Impacted Connection Count": 1,
            "Subdomain Count": 2,
            "Impact/Total (%)": 44.6
        },
        {
            "Domain": "example2.com",
            "Total Connection Time (ms)": 170,
            "Impact Time (ms)": 55,
            "Impacted Connection Count": 1,
            "Subdomain Count": 2,
            "Impact/Total (%)": 32.4
        }
    ]
}

Retrieve the most significant Issue by Issue Burst

Route Method
/v0/sessions/analysis/impact POST

This endpoint is deprecated since November 2019. Please use the Retrieve a summary of issues endpoint instead.

Get the session video metadata

Route Method
/v0/sessions/{session_id}/video/metadata GET

Example


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/{session_id}/video/metadata

Response

If the session video has audio:


{
    "session_id": "862210fb-7c90-11ed-bb00-06c4c57ba6cd",
    "dimensions": {
        "height": 800,
        "width": 448
    },
    "fps": 29.443,
    "video_duration_ms": 87796,
    "video_codec": "h264",
    "video_container_format": "mp4",
    "contains_audio": true,
    "audio_num_channels": 2
}

If the session video does not have audio:


{
    "session_id": "dc5383f9-969b-11ed-a572-023bc66c5bad",
    "dimensions": {
        "height": 800,
        "width": 376
    },
    "fps": 30.588,
    "video_duration_ms": 21773,
    "video_codec": "h264",
    "video_container_format": "mp4",
    "contains_audio": false,
    "audio_num_channels": null
}

Session Time Series Data

Retrieve Available Session Time Series

Download Session Time Series

Retrieve Available Session Time Series

Route Method
/v0/sessions/timeseries/{session_id}/info GET

Examples

To save the retrieved data as files on your computer, use the flag <code class="dcode">-o {filepath}</code> in your <code class="dcoed">cURL</code> commands.


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/timeseries/{session_id}/info

To output the dictionary to a file, use the following:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/timeseries/{session_id}/info -o "time_series_config.json"

Response

The response is a json dictionary of attributes for all available time series classes (in a given session). This dictionary contains a mapping of time_series_key -> time_series_name, time_series_category, and time_series_units.


{
    "impact": {
        "name": "Impact Count",
        "category": "impact",
        "units": "impacts",
    },
    "memory_used": {
        "name": "Memory Used",
        "category": "device",
        "units": "MiB",
    }
}

Download Session Time Series

Route Method
/v0/sessions/timeseries/{session_id}/download?key={time_series_key} GET

Required Parameters

The <code class="dcode">key</code> parameter is used to retrieve data for that particular time series.

Examples

To save the retrieved data as files on your computer, use the flag <code class="dcode">-o {filepath}</code> in your <code class="dcode">cURL</code> commands.


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/timeseries/{session_id}/download?key={time_series_key}

To output to a file (i.e. a csv), use the following:


curl https://<your_api_token>@api-dev.headspin.io/v0/sessions/timeseries/{session_id}/download?key={time_series_key} -o "time_series_data.csv"

Response

The response is the time series data for a particular time series and session in CSV format.


,Time,Impact Count,Units
0,0,0.0,impacts
1,19,0.0,impacts
2,25,0.0,impacts
3,19160,1.0,impacts
4,19200,0.0,impacts
5,25985,1.0,impacts
6,26256,0.0,impacts
7,27090,1.0,impacts
8,27193,2.0,impacts
9,27225,0.0,impacts
10,27500,3.0,impacts
11,27521,1.0,impacts
12,27529,2.0,impacts
13,27582,0.0,impacts
14,27960,1.0,impacts
15,28920,0.0,impacts
16,36105,3.0,impacts
17,36130,1.0,impacts
18,36203,0.0,impacts
19,38120,1.0,impacts
20,38740,3.0,impacts
21,38761,2.0,impacts
22,38836,1.0,impacts
23,39960,0.0,impacts
24,40222,2.0,impacts
25,40247,1.0,impacts
26,40303,0.0,impacts
27,50075,1.0,impacts
28,50391,0.0,impacts