Image Captioning Service in Filestack

Overview

As an intelligence service of the Filestack platform, Image Captioning is able to describe your images, and return captions synchronously or even through the Workflows.

Resources

Processing API

Image Captioning is available as a synchronous operation in the Processing API using the following task:

caption

In order to use this task in Processing API, you must use security policy and signature. Read more about security policies here.

Response

{
    "caption": "a close up of a bird"
}

Response Parameters


caption	string	The caption of the image

Examples

Get the caption of your uploaded image with its handle:

https://cdn.filestackcontent.com/security=p:<POLICY>,s:<SIGNATURE>/caption/<HANDLE>

Use caption in a chain with other tasks, e.g. resize:

https://cdn.filestackcontent.com/security=p:<POLICY>,s:<SIGNATURE>/resize=h:1000/caption/<HANDLE>

Use caption with an external URL:

https://cdn.filestackcontent.com/<FILESTACK_API_KEY>/security=p:<POLICY>,s:<SIGNATURE>/caption/<EXTERNAL_URL>

Use caption with Storage Aliases:

https://cdn.filestackcontent.com/<FILESTACK_API_KEY>/security=p:<POLICY>,s:<SIGNATURE>/caption/src://<STORAGE_ALIAS>/<PATH_TO_FILE>

Workflows Task Configuration

Image Captioning task is available in the Intelligence tasks category in Filestack Workflows. To learn more about how to use the Workflows UI for configuring tasks (and the logic between them), visit Creating Workflows Tutorial.

Workflows Parameters


Task Name	string	Unique name of the task. It will be included in the webhook response and can be used to build the logic below.

Logic

Image Captioning task returns the following response:

{
    "data": {
        "caption": "the description of the image"
    }
}

Logic Parameters


data	dictionary	Indicates the response from this service.
caption	string	The description of the image

Based on the task’s response you can build logic that tells the workflow how dependent tasks should be executed. For example, if you would like to run another task if a specific task was detected you can use the following rule:

caption kex "bird"

In Workflows UI this command would look like in the example below:

Visit Creating Workflows Tutorial to learn how you can use Workflows UI to configure your tasks and logic between them.

Webhook

Below is an example webhook payload for an image captioning task. “caption” is the detected description of the image.

{
    "id": 67379682,
    "action": "fs.workflow",
    "timestamp": 1569862391,
    "text": {
        "workflow": "f2062def-2859-4529-a13a-4b3069362e45",
        "createdAt": "2019-09-30T17:18:10.801570624Z",
        "updatedAt": "2019-09-30T17:18:12.433019879Z",
        "sources": [
            "3PnPnQyYTqaSdQLo8Cxq"
        ],
        "results": {
            "caption_1569585835604": {
                "data": {
                    "caption": "a close up of a bird"
                }
            }
        },
        "status": "Finished"
    }
}

Please visit the webhooks documentation page to learn more.