Skip to content

DataLab Notebook Runner

Overview

The DataLab Notebook Runner module is for remotely running your DataLab notebook files in a Composable DataLab using Composapy and Papermill. It takes a DataLab ID, a name of the notebook to run, a list of string-object pairs as parameters, and returns a list of string-object pairs.

Example

Setup


!DataLabs Notebook Example

1) Create a new DataLab, naming it "An Overly Complicated Calculator"

2) Create a new notebook, naming it "add.ipynb".

3) Using the logic for parameterizing from the Parameterize docs page, tag the first cell (a = 1, \n b = 1) as "parameters". This cell's parameters will be overridden by the inputs passed in from the DataFlow Notebook Runner module.

return_values : dict

The keyword return_values is used by Composable to send your results back to the DataFlow Notebook Runner module.

  • Each key must be a string

  • A restricted number of types are supported for values:

    • int
    • str
    • None
    • CompAnalytics.Contracts.FileReference (although FileReference may be passed to and from a notebook, any file references created in the notebook context will be unaccessible by the DataFlow)

DataFlow


!DataLabs Notebook Runner DataFlow Example

1) Create or import the DataFlow as seen above. - Click the pencil icon and choose the DataLab named "An Overly Complicated Calculator" - In the notebook input, include the relative path to your notebook (in this case, the notebook add.ipynb was contained within the custom calculator subdirectory) - When passing inputs into parameters, use an object namer module to preserve your input types

2) Run the DataFlow. Your results window should contain the return_values passed from your add.ipynb notebook.

Notes


Although not officially supported, you can pass other csharp types back with pythonnet by importing loading composapy into your notebook session.

import composapy

You can then import C# and Composable types as python objects.

# example imports
import System
from CompAnalytics.Contracts import *

To learn more about this type of behavior, check out the pythonnet github.