# Workflows

# Using Workflows

Workflows are a convenient method for running **graphs of asset-processing jobs**.

![Workflow graph example](https://storage.googleapis.com/metafold_static-1/docs/workflow_graph.png)

Workflows are configured in [YAML][]. Here is an example of a workflow definition for
converting a tessellated mesh file to a signed distance field (SDF) sampled onto a grid:

```yaml
---
jobs:
  preprocess-mesh:
    type: mesh/preprocess
  compute-bvh:
    type: mesh/compute-bvh
    needs:
    - preprocess-mesh
    assets:
      mesh: preprocess-mesh
  sample-mesh:
    type: implicit/from-mesh
    needs:
    - preprocess-mesh
    - compute-bvh
    assets:
      mesh: preprocess-mesh
      bvh: compute-bvh
...
```

Workflows are composed of a graph of asset-processing jobs. Each job may receive a
number of input assets and parameters and will often output at least one asset and some
accompanying parameters. Input/output details for each job may be found on the [workflow
dashboard][job types].

Workflow configurations are intended to establish dependency connections between jobs,
leaving only required input assets and parameters to be specified at dispatch. For
example, the workflow definition above connects the `mesh` output from `preprocess-mesh`
to both `compute-bvh` and `sample-mesh`. The job `mesh/preprocess` itself receives an
input mesh asset, which is specified when making a `POST` request to dispatch the
workflow.

## Dispatching workflows

```js
const body = JSON.stringify({
  // YAML-encoded configuration
  definition: "...",
  assets: {
    // Filename of uploaded asset
    "preprocess-mesh.mesh": "part.stl",
  },
  parameters: {
    "preprocess-mesh.center": "bounding_box",
    "sample-mesh.resolution": "512",
  },
});

fetch("...", {
  headers: {
    "Content-Type": "application/json",
    // ...
  },
  body,
});
```

Assets/parameters may be assigned at dispatch by specifying the job name and
corresponding asset/parameter name separated by a `.` character.

When the workflow is complete, users are free to then query job inputs and outputs using
the usual job querying endpoints to retrieve any intermediate/resulting assets and
computed parameters.

While configurations may be dynamically generated as needed, clients gain the most
mileage through **repeated usage** of **preset configurations**. This is not dissimilar
to declaratively-configured continuous integration and deployment (CI/CD) setups (e.g.
GitLab, GitHub), but instead of files checked-in to Git, our inputs are assets that have
previously been uploaded to the Metafold REST API.

### Additional notes

- Parameters must be encoded as strings, e.g. `"[0.0, 0.0, 1.0]"` for a list with 3
  float values

## YAML Syntax Reference

This section explains the YAML syntax used to configure workflows. See some common
workflow examples in our [Python SDK][examples].

### Root keywords

These keywords live at the root of the YAML document.

#### `jobs`

Mapping from **unique**, user-defined names to job configurations.
See [Job keywords](#topic-job-keywords).

- Job names **must not** contain any `.` period characters

We recommend giving each job a short, descriptive name to indicate its role in the
workflow (which may not always be obvious from the job type).

### Job keywords

The following headings describe the keywords required to configure workflow jobs.

#### `type`

Job type. The full list of available job types and their input/outputs may be found on
your [workflow dashboard][job types].

#### `needs`

List of **upstream dependencies** for this job. If a job has any upstream dependencies,
the required output assets or parameters from the upstream job must be specified in the
assets or parameters mapping.

#### `assets`/`parameters`

Mapping between input assets/parameters from this job, to output assets/parameters from
any upstream job listed in `needs`. Upstream assets/parameters may be specified in
**short form** if the upstream job has only **one** output, e.g.:

```yaml
assets:
  mesh: upstream-job
```

If the upstream job has **more than one** output, the mapping must be provided in **long
form** to disambiguate between outputs. In long form, the mapping must specify both the
job name and the output asset/parameter name. For example:

```yaml
assets:
  mesh:
    job: upstream-job
    asset: result
```

[YAML]: https://yaml.org/
[examples]: https://github.com/Metafold3d/metafold-python/tree/master/workflows
[job types]: https://account.metafold3d.com/workflows/jobs