Valossa Core API and Valossa Training API documentation (1.0.0)

HTTP POST. Create a new video analysis job for Valossa AI. The job will be processed after a while when the system has picked it for analysis. Of course, the job is not finished yet when the new_job call returns. A successful return just indicates that the job request was accepted. For actual analysis status you will use the job_status function, and for analysis results you will use the job_results function.

There are two different ingestion methods for your video file — download and upload. Download is clearly the more straightforward method to use, if you are working via the API.

Downloading involves a URL that points to your downloadable video file, and you use the download URL in your new_job request. When using a download URL, provide a direct(!) link to a video file(!). We will not scrape any videos from a HTML page. You must provide an actual video, not a web page that happens to contain a video as part of it. Please note: the system downloads the files from your download URLs in an asynchronous manner, so there will be some delay between the creation of the job and the initiation of the download. Corollary: you shouldn't use expiring URLs, or, if you really need to use them, you should make the expiry time very long. Moreover, you shouldn't input an unreasonable number of files per day.

Uploading your video file via the graphical user interface of Valossa Portal is easy, and that is probably the first file ingestion method you try out when familiarizing yourself with Valossa AI. Locate the file on your local hard drive and use the "Add Videos" button to send the file to analysis.

Uploading your video file via the API involves first the use of the upload functionality of the Valossa Video Recognition API. After the upload has been completed, you use the resulting valossaupload:// URL in your new_job request to refer to the specific uploaded file. There are 3 different API requests that make it easy to upload your file. Because videos often are huge, the files must be uploaded in chunks. In order to learn how to upload a video file, start by reading the documentation of the API function "initialize_file_upload".

In the download video ingestion method, the video URL and transcript URL can be either http:// or https:// or s3:// based. If the URL is s3:// based, you should first communicate with us to ensure that our system has read access to your S3 bucket in AWS (Amazon Web Services).

Whether you used the download or upload video ingestion method, the video URL is mandatory in the new_job request. The URL must directly point to a downloadable video file (in which case, our system will download the file from your system) or it must be a valossaupload:// URL of your uploaded file.

The transcript URL is optional – but recommended, because an existing SRT transcript is a more reliable source of speech information than audio analysis. The URL must directly point to a downloadable SRT transcript file. Our system will download the file from your system.

The title is optional – but recommended. A human-readable title makes it easy for you to identify the video on the results page (https://portal.valossa.com/portal/home) of Valossa Portal, and will also be included in the metadata file.

The description is optional. Description is any freetext, in English, that describes the video.

If title and/or description are provided in the call, the text in them will also be analyzed, and the detected concepts will be included in the analysis results (the "external" concepts in the metadata JSON).

The customer media info is optional. If you provide a customer media ID in the "id" field inside the "customer_media_info" field, you may use the customer media ID (a string from your own content management system) to refer to the specific job in the subsequent API calls, replacing the "job_id" parameter with a "customer_media_id" parameter in your calls. Our system will NOT ensure that the customer media ID is unique across all jobs. Duplicate IDs will be accepted in new_job calls. It is the responsibility of your system to use unique customer media IDs, if your application logic requires customer media IDs to be unique. If you use duplicate customer media IDs, then the latest inserted job with the specific customer media ID will be picked when you use the "customer_media_id" parameter in the subsequent API calls.

The language is optional. It specifies the language model to be used for analyzing the speech in the audio track of your video. The allowed values are listed on the page https://portal.valossa.com/portal/docs#/languages

If the language parameter is not given, the default "en-US" will be used so the speech in the video is assumed to be in US English.

Please note that for other languages than US English, the following exceptions apply.

• Audio-based detections of named entities ("audio.keyword.name.person", "audio.keyword.name.location", "audio.keyword.name.organization", "audio.keyword.name.general" detections) and offensive words ("explicit_content.audio.offensive" detections) are available, but audio-based detections of novelty words ("audio.keyword.novelty_word" detections) are not available.
  • However, for the French language, "audio.keyword.novelty_word" detections are available (as English translations).
• Audio-based IAB categories are not available.

If the analysis is technically successful (i.e. if the job reaches the "finished" state), the job will be recorded as part of the used volume on your ongoing subscription period.

In the examples section, you can also find an example new_job request body with only the mandatory fields present. Moreover, in the examples section you can also find an example new_job request body with the specification to use a non-default, self-created face gallery (managed by using the Valossa Training API), so the faces in that gallery will be used for identifying the persons in the video.

If you have a default face gallery and want to use it in your video analysis job, just leave out the gallery selection in the new_job call. The default face gallery will be implicitly selected, when no other gallery is explicitly selected.

If you want a HTTP POST callback and/or email notification when your video analysis job reaches an end state, you may specify one or both of those in the new_job request.

The HTTP POST callback mechanism in our system expects your system to send a 200 OK response for the request (callback) initiated by our system. The request will be re-tried one time by our system, if the first attempt to access your specified callback URL returns a non-200 code from your system or times out. Due to the possibility of network problems and other reasons, you should NOT rely on the HTTP POST callback to be received by your system. In any case, whether the HTTP POST callback event was received or not, your system can always check the status of the job using the job_status function in the REST API. In the examples section, you will find an example new_job request for specifying a callback. Here is an example of a HTTP POST callback message body: {"job_id": "ad48de9c-982e-411d-93a5-d665d30c2e92", "status": "finished"}

The email notification will be sent to those users that have the permission to view job results for the chosen API key. The generated email notification message is intended for a human recipient. So, unlike the HTTP POST callback, the email notification message is not intended for machine parsing. In the examples section, you will find an example new_job request with an email notification specified. Currently, the only supported value for the "to_group" parameter is "users_with_access_to_api_key_results".

Please see the various new_job examples, which are selectable from the Example dropdown-menu-box on the side. However, for your convenience, below is a dummy new_job request that shows a large number of fields (including non-mandatory fields) that can be used in a new_job request — yes, this is overly complex for actual use but gives you a quick glance over the possible data fields:

{
  "api_key": "kkkkkkkkkkkkkkkkkkkk",
  "media": {
    "title": "The Action Movie",
    "description": "Armed with a knife, Jack Jackson faces countless dangers in the jungle.",
    "video": {"url": "https://example.com/content/Action_Movie_1989.mp4"},
    "transcript": {"url": "https://example.com/content/actionmovie.srt"},
    "customer_media_info": {"id": "m_0717286314-A"},
    "language": "en-US"
  },
  "analysis_parameters": {
    "face_galleries": {"custom_gallery": {"id": "468a2b70-3b55-46f8-b209-8ad2fcabd5c8"}}
  },
  "email_notification": {
    "to_group": "users_with_access_to_api_key_results"
  },
  "callback": {
    "url": "https://example.com/cb"
  }
}

The status of a single analysis job is polled using this request. Possible values for the "status" field in the response are "queued", "on_hold", "preparing_analysis", "processing", "finished", and "error". More status values may be introduced in the future.

If the job status is "error", something went wrong during the analysis process. If there is an explanation of the error in the "details" field, please see if the cause of the error is something you can fix for yourself (such as a non-video file in the video URL of the job request). Otherwise, contact us in order to resolve the issue.

If the job status is "queued" or "processing", you should poll the status again after some time.

If the job status is "finished", you can fetch the job results using the job_results function.

The "details" field may contain some additional details about the status of the job.

The "media_transfer_status" field indicates whether the media to be analyzed has been transferred from your system to our system. Possible values for the "media_transfer_status" field: "queued", "downloading", "finished" and "error". If "media_transfer_status" is "finished", your video (and the transcript if you provided it) have been successfully transferred to our system.

The value in "poll_again_after_seconds" is just a suggestion about when you should poll the job status again (expressed as seconds to wait after the current job_status request).

If there was a problem with the job_status query itself, the error will be indicated in an HTTP non-201 response with a JSON body, similar to the error responses of the new_job function.

After a job has been finished, the resulting video metadata can be fetched using this request. Response data is in the Valossa Metadata JSON format. For more details on the data format, see the documentation chapter https://portal.valossa.com/portal/docs#/metadatageneral?id=basics-of-reading-valossa-metadata-in-your-application

Save the Valossa Metadata and use it from your own storage disk or database for your easy and quick access. We will not necessarily store the results perpetually.

The optional GET parameter "type" can be used for requesting a specific type of results data. The default value for "type" is "core", which stands for Valossa Core metadata. A specific type of results data is available only if it was generated for your video analysis job, and this obviously depends on the details of the Valossa subscription that you are using. The possible values for the "type" parameter are "core", "frames_faces", "frames_objects", "seconds_objects", "speech_to_text_srt", "speech_to_text_vtt", "speech_to_text_ttml", "speech_to_text_avid_txt" and "visual_captions".

If there was a problem with the job_results query itself, the error will be indicated in an HTTP non-200 response with a JSON body, similar to the error responses of the new_job function.

Want to search for specific recognized things among your accumulated video analysis job results, by typing search words such as "airplane" or "Emilia Clarke"? Please try the easy-to-use Search functionality in Valossa Portal.

Convenience function for listing all your jobs, optionally with also their job statuses (optional parameter "show_status" with the value "true").

If you had given a customer media ID when creating the job, the "customer_media_id" field exists in the response and contains the customer media ID value.

Showing video titles and other media information in the job listing is often useful. This can be done by using the optional GET parameter "show_media_info" with the value "true". The examples section shows also an example about that.

By adding the optional GET parameter "n_jobs" to the request (example: n_jobs=500), you can control how many of your jobs will be listed if your job list is long. The default is 200. The maximum possible value for "n_jobs" is 10000.

If there was a problem with the list_jobs query itself, the error will be indicated in an HTTP non-200 response with a JSON body, similar to the error responses of the new_job function.

Cancel a job. The job must be in a cancellable state for this function to succeed. For example, a finished job is not cancellable.

If there was a problem with the cancel_job query itself, the error will be indicated in an HTTP non-200 response with a JSON body, similar to the error responses of the new_job function.

Delete a job. Deleting a job will remove it from the set of your existing jobs. When a job is deleted, the assets (video file etc.) associated with the job are also deleted. There may be some delay between completing the delete_job query and the purging of all assets from the storage systems in the background.

If there was a problem with the delete_job query itself, the error will be indicated in an HTTP non-200 response with a JSON body, similar to the error responses of the new_job function.

Initialize a video file upload.

Providing a video file download URL in "new_job" is much more straightforward, and download is thus recommended instead of using an upload. However, sometimes a download link cannot be hosted by our customer, and so the API also supports uploading of video files.

There are 3 different API requests that make it easy to upload your file, and this request is the first of those 3 to be sent. You initialize the upload with this function. Because videos often are huge, the files must be uploaded in chunks.

You must specify the size of the file to be uploaded (in bytes), and the response will tell you the chunk size (in bytes) that you must use when sending your file chunks. Save the upload ID (from the "upload_id" field of the response) in order to be able to refer to the same upload action in the subsequent requests.

The Content-Type header of the initialize_file_upload request must be "multipart/form-data". If you use Curl and its "-F" option, Curl will set this Content-Type as default and will also use POST as the HTTP method in the request.

Next, upload each chunk of your video file by calling the "send_file_chunk" function repeatedly. The size of each chunk must be exactly the number of bytes indicated in the "file_chunk_size_bytes" field in the response you received for your initialize_file_upload request. However, the last chunk of the file may have a different size; this is natural, because the total size of the file usually is not an exact multiple of the chunk size.

How to split your video file into chunks? In Linux-style operating systems, the "split" command is suitable for the task. We recommend using the "-d" option of the "split" command, because then the chunks will be named so that the chunk index in the chunk file names is a number sequence rather than a letter sequence that is the default. The number-based indexing is probably easier to use in your own helper scripts.

Split example:

split -d --bytes=4194304 supervideo.mp4 supervideo.mp4.chunk

The above command creates the files supervideo.mp4.chunk00, supervideo.mp4.chunk01, supervideo.mp4.chunk02 with a size of 4194304 bytes and the file supervideo.mp4.chunk03 with a size of 157927 bytes.

Send a chunk of the video file that you are uploading. This function is called repeatedly: each chunk is sent in a separate request.

How to split your video file into chunks correctly and conveniently? See instructions for that in the documentation of the "initialize_file_upload" function.

When uploading file chunks, you must specify the upload ID and chunk index. The chunk index is an integer, and the first chunk of the file has the index 0. The temporal order of the send_file_chunk requests with the different chunk indexes does not need to be 0, 1, 2... but you must specify the correct index for each chunk, so the uploaded file will be correct when the chunks are reassembled into the full file in our system. Of course, probably it is easiest for you to use a helper script that just sends the chunks in the order 0, 1, 2..., increasing the value of chunk_index in a loop.

The Content-Type header of the send_file_chunk request must be "multipart/form-data".

When all chunks have been uploaded, you need to finalize the upload by using the "finalize_file_upload" function.

Finalize a video file upload. When all chunks have been uploaded with the "send_file_chunk" function, you need to finalize the upload, referring to the correct upload ID.

The Content-Type header of the finalize_file_upload request must be "multipart/form-data".

Save the valossaupload:// URL from the "uploaded_file_url" field of the response, so you can use it in your new_job request (in the "media":"video":"url" field) for that video.

For an entirely manual upload of a file in a graphical user environment, use the Analyze page in Valossa Portal.

Works only with an API key that is bound to a faces-enabled Valossa subscription.

Upload a face image for training.

Adding sample images for a face has been designed to work with both, file uploads and file downloads. Thus, the file reference mechanism used in the add_face_image request of the Valossa Training API uses an easy URL-based syntax for both file input styles. Currently, only uploads are supported, but download support will be added in the future. Download, obviously, means that our system downloads the image file from an HTTP(S) URL provided by your system. Uploaded files each get assigned a valossaupload:// URL that uniquely identifies the successfully received file that resides in our storage system.

When you upload a file, first use the REST function upload_image to move the file content. Then, you will use the valossaupload:// URL of the correct file when referring to the file in the add_face_image request in the actual face identity training.

Before using the service in a way that makes you a "processor" and/or "controller" of the personal data of EU-resident natural persons, you are required to make sure that your actions are compliant with the General Data Protection Regulation. See the Terms and Conditions of Valossa services at https://valossa.com/terms-conditions/

An image must be in the JPG or PNG format. The maximum image file size is 8MB. The maximum width of an image is 4096 pixels. The maximum height of an image is 4096 pixels.

At least 10 different sample images of each face, photographed from different angles etc., should be given in order to get good training results. The more images the better. Training may in some cases work even with only a few images, but the results are better with more samples: a lot of clear, diverse, high-quality images of the face to be trained.

The upload_image call is similar regardless of the gallery type (default vs. non-default), because this request does not operate on a gallery; it just uploads an image for further use that probably involves a gallery.

The Content-Type header of the file upload request must be "multipart/form-data". If you use Curl and its "-F" option, Curl will set this Content-Type as default and will also use POST as the HTTP method in the request. There must be one file per upload_image request.

Note! As you can see from the Curl request example, the API key must be sent as a form parameter (not URL parameter). This is quite natural, taking into account that the Content-Type of the request is "multipart/form-data".

Works only with an API key that is bound to a faces-enabled Valossa subscription.

Create a new face gallery. You may decide to use your default face gallery that is created implicitly when you start training faces without specifying a face gallery. In that case, please skip face gallery creation. Otherwise, please explicitly create your non-default face gallery by using this request.

Save the gallery ID locally. You will need it when you add face identities to the gallery or when you do any other operations with the specific gallery.

The maximum length of the "name" parameter of a face gallery is 1024 characters.

How to specify your custom face gallery (by its gallery ID, a unique identifier) to be used in an analysis job? In the instructions for the new_job API function, please see the request example for using a custom_gallery structure inside a face_galleries structure inside an analysis_parameters structure.

Works only with an API key that is bound to a faces-enabled Valossa subscription.

Update a custom face gallery. The "updates" structure contains the face gallery parameters to update. Currently the only allowed parameter is "name". The data type for this value is string. The maximum length of the value of "name" is 1024 characters.

Only a non-default face gallery can be updated. The default face gallery cannot be modified by this request. If you need named galleries, you should be using non-default, explicitly created face galleries (create such a gallery by using the "create_face_gallery" function).

Works only with an API key that is bound to a faces-enabled Valossa subscription.

List your existing face galleries. It is also possible to include the details of your existing face galleries in this listing by providing the optional GET parameter "show_details=true".

Works only with an API key that is bound to a faces-enabled Valossa subscription.

Create a new face identity. The string fields "name" and "gender" are optional. We recommend setting at least the name, because a nameless face identity might cause confusion for you later on (however, it is perfectly acceptable to have a nameless face identity, if your application logic requires creating such an identity). The maximum length of the value of "name" is 1024 characters. The gender is "male" or "female". The response contains the unique identifier of the face identity (person).

Save the face ID locally. You will need it when you add images for the face or when you do any other operations with the specific face identity.

Works only with an API key that is bound to a faces-enabled Valossa subscription.

Add an image to a face identity. Referring to your previously uploaded image files, you need to add the correct files to a specific existing face identity, one image file per add_face_image request. The response contains a unique identifier of the processed, accepted training image, from which a sample face has been detected. You need the ID later, if you want to do any operations with this training image that has been added to a specific face identity.

There must be exactly one face visible per image. This REST function may take a few seconds to complete, because the system checks that exactly one face is clearly visible (otherwise, an error response is generated).

In the future, also image download URLs will be able to be used with the same easy add_face_image call syntax. Currently, only the valossaupload:// URLs created as a result of file uploads are supported.

Please make sure that each of the images is actually an image of the correct person. Typically, checking this involves some human work. Wrong images will deteriorate the quality of face detections.

The image needs to be a good-quality frontal or nearly-frontal face image.

In this request, there is no need to specify the gallery, even when using a non-default gallery. The face ID is a unique identifier for the correct face, so the correct gallery is implicitly used.

Works only with an API key that is bound to a faces-enabled Valossa subscription.

List existing face identities in a gallery. It is also possible to include the details of your existing face identities in this listing by providing the optional GET parameter "show_details=true".

Works only with an API key that is bound to a faces-enabled Valossa subscription.

List existing images added for a specific face identity. In this request, there is no need to specify the gallery, even when using a non-default gallery. The face ID is a unique identifier for the correct face, so the correct gallery is implicitly used.

Works only with an API key that is bound to a faces-enabled Valossa subscription.

Update an existing face identity. In this request, there is no need to specify the gallery, even when using a non-default gallery. The face ID is a unique identifier for the correct face, so the correct gallery is implicitly used.

The "updates" structure contains one or more face parameters to update. The allowed parameters are "name" and "gender". The data type for these values is string. The maximum length of the value of "name" is 1024 characters. The value for "gender" is "male" or "female".

Note. To unset a field such as "name" or "gender" completely, just set it to null in an update_face_identity call. In an update, a value that is not mentioned in the "updates" structure will retain its old value if it had one (in other words, omitting the field from the update does not unset the value of the field, while setting it explicitly to null will unset it).

Works only with an API key that is bound to a faces-enabled Valossa subscription.

Remove a face image from a specific face identity. In this request, there is no need to specify the gallery, even when using a non-default gallery. The image ID is a unique identifier for the correct image, so the correct gallery is implicitly used.

Works only with an API key that is bound to a faces-enabled Valossa subscription.

Remove an existing face identity. In this request, there is no need to specify the gallery, even when using a non-default gallery. The face ID is a unique identifier for the correct face, so the correct gallery is implicitly used.