Functions turn a local Function project into a named, operated process. You define the project in function.py: source bundle, install command, entrypoint, env var names, optional service port, declared outputs, and runtime policy. Nullspace creates Function versions, job runs, service instances, logs, output metadata, and backing machines. Use this path when the agent is part of your product workflow and your application wants to call it repeatedly. Your backend stays the control plane for users, auth, billing, product data, and business rules. Nullspace is the compute plane for the agent process: isolated machines, deploy/run records, endpoint traffic, logs, outputs, and retained debugging access. Current live modes are job and service. Nullspace run IDs are not conversation session IDs.

Lifecycle

  1. Create function.py with nullspace functions init.
  2. Select project files with bundle=nullspace.Bundle(...) when inference is not enough.
  3. Iterate locally with nullspace dev.
  4. Deploy the project with nullspace deploy.
  5. Run a finite job or start a service endpoint.
  6. Inspect logs, status, outputs, and the backing machine.
  7. Retain failed machines when you need PTY, SSH, desktop, command, or file debugging.
  8. Deploy a new version, roll back, or delete the Function when it is no longer needed.

Feature Surface

AreaWhat is available
Authoringfunction.py, nullspace.Function(...), Python and TypeScript projects, explicit bundles, custom templates, warmup hooks, resources, policy, debug retention, outputs, and max concurrency.
Deploymentnullspace deploy, nullspace functions deploy, dry-run bundle plans, build cache controls, version history, git SHA provenance, and forward-only rollback.
Jobsrun, spawn, map, route selection, runtime env overrides, idempotency keys, bounded concurrency/backpressure, logs, outputs, cancellation, and backing machine access.
ServicesService URL lookup, readiness checks, public URL policy, runtime env overrides, restart/stop, logs, checkpoint/branch, and backing machine access.
Routing and triggersNamed routes, cron schedules, period schedules, inbound HMAC webhooks, trigger fire history, and route-specific trigger targets.
Security and environmentsNamed Function secrets, request-time env values, dev/staging/prod environments, edge proxy-auth tokens for service endpoints, and app-owned bearer-token checks.
OperationsList/get/delete Functions, inspect status, follow logs, retain failed machines, checkpoint runs/services, branch runs/services, lineage trees, rollback, and Machine+Function composition.

Setup

From a local project directory:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install "nullspace-sdk[cli]"
nullspace auth login --api-url https://18.140.200.84.sslip.io

nullspace functions init
nullspace dev
nullspace deploy --dry-run --json
nullspace deploy --json
If uv is already part of the project, uv pip install "nullspace-sdk[cli]" is equivalent after the environment is active. For CI or non-interactive deploys, use NULLSPACE_API_KEY or NULLSPACE_TOKEN with NULLSPACE_API_URL instead of auth login.

How Functions Fit Your App

For a product backend, a Function is a named agent endpoint. The backend stores the Nullspace Function name, calls the SDK or API with a server-side API key, and maps application concepts like user IDs, tasks, or conversations to Function run IDs or service instance IDs. Use a job for one-shot work:
frontend -> your backend -> Nullspace Function run -> outputs or callback
Typical job work includes repository review, report generation, data extraction, evaluations, or a framework app that exits after producing artifacts. The run has logs, status, outputs, and a backing machine that can be retained for debugging. Use a service for a long-running HTTP process:
frontend -> your backend -> Function service URL -> function service process
Typical service work includes a LangGraph API, a chat backend, a preview server, or a custom coordinator. Your app still owns product sessions, authorization, and durable state. Nullspace owns the isolated process, endpoint routing, service logs, and machine access. Keep framework state where the framework expects it. LangGraph thread IDs, Claude sessions, OpenAI Agents SDK traces, CrewAI memory, and custom conversation records are app-owned unless you intentionally place them in a retained machine, mounted volume, or external database.

Environments And CI

Function environments scope Function names, named secrets, version history, and name-based run/spawn/rollback lookups. Omitted environment selectors use default.
nullspace environment create staging
nullspace environment list --json

NULLSPACE_ENVIRONMENT=staging nullspace deploy --json
nullspace functions list --env staging --json
nullspace functions history repo-review-agent --env staging --json
nullspace run repo-review-agent --env staging --json
--env NAME=value remains a runtime env-var override. A bare --env staging or --environment staging selects the Function environment for commands that resolve a Function by name. CI jobs can set NULLSPACE_ENVIRONMENT from the branch or workflow matrix and run the same nullspace deploy command for dev, staging, and prod. Deploy records the local git SHA on the Function version when the checkout has one; nullspace functions history and the console version table show that provenance. For a complete GitHub Actions workflow, see examples/github-actions/functions-deploy.yml in the repository.

What You Deploy

nullspace functions init writes function.py. Keep runtime secret values out of this file; store env var names, named secret references, and non-secret configuration. For a finite job:
function.py
import nullspace

function = nullspace.Function(
    "repo-review-agent",
    mode="job",
    template="base",
    workdir="/workspace/project",
    install="python -m pip install -e .",
    entrypoint="python main.py",
    env=nullspace.Env(
        required=["OPENAI_API_KEY"],
        optional=["SERPER_API_KEY"],
    ),
    secrets=[nullspace.Secret.from_name("openai")],
    outputs=nullspace.Outputs(paths=["result.json", "reports"]),
    max_concurrency=4,
)
For a long-running service, set mode = "service" and bind the process to 0.0.0.0:
function.py
import nullspace

function = nullspace.Function(
    "repo-review-service",
    mode="service",
    template="base",
    workdir="/workspace/project",
    install="python -m pip install -e .",
    entrypoint="python -m uvicorn app:app --host 0.0.0.0 --port 8000",
    service=nullspace.Service(
        port=8000,
        readiness={"type": "http", "path": "/health", "timeout_seconds": 30},
    ),
    policy=nullspace.Policy(public_url=True),
)

Definition Fields

The function.py descriptor is the source of truth for deploy behavior:
FieldPurpose
name / modeStable Function name and job or service runtime shape. If mode is omitted, a service block or routes infer service mode; otherwise the default is job.
template / workdirMachine template or template builder plus the working directory inside the guest.
install / warmup / entrypointDependency install command, optional deploy-time warmup command, and runtime command.
bundleIncluded and excluded local files for upload.
env / secretsRequired/optional env var names and named secret references. Values stay out of function.py.
resources / policyCPU, memory, disk, timeout, internet/public URL, egress, volume, desktop, PTY, debug, and maximum resource policy.
outputs / debugOutput files/directories to publish after jobs and failed-run machine retention defaults.
serviceService port, readiness check, and idle timeout for long-running HTTP processes.
routes / schedulesNamed entrypoints and cron/period triggers.
max_concurrency / requires_proxy_authPer-Function admission cap and edge-enforced endpoint credentials for public services.
function.py
import nullspace

function = nullspace.Function(
    "repo-review-agent",
    mode="job",
    template="base",
    workdir="/workspace/project",
    install="python -m pip install -e .",
    entrypoint="python main.py",
    bundle=nullspace.Bundle(include=["src/**", "pyproject.toml"], exclude=[".env"]),
    resources=nullspace.Resources(vcpus=2, memory_mb=2048, timeout_seconds=900),
    policy=nullspace.Policy(internet_access=True, allow_out=["api.github.com"]),
    debug=nullspace.Debug(retain_on_failure=True, retain_seconds=3600),
    outputs=nullspace.Outputs(paths=["result.json", "reports"]),
    max_concurrency=4,
)

Named Routes

Routes let one Function expose multiple job entrypoints or endpoint paths while sharing the same bundle, version history, secrets, templates, and environment. Every route path starts with /. A route can reuse the default entrypoint or override it with a route-specific command.
function.py
import nullspace

function = nullspace.Function(
    "repo-agent",
    mode="job",
    install="python -m pip install -e .",
    entrypoint="python -m agent.default",
    routes=[
        nullspace.Route("/review", entrypoint="python -m agent.review"),
        nullspace.Route("/report", entrypoint="python -m agent.report"),
    ],
    schedules=[
        nullspace.Cron("0 8 * * 1", route="/report"),
        nullspace.Period(seconds=21600, route="/review"),
    ],
)
Select routes from the CLI, Python SDK, TypeScript SDK, schedules, webhooks, and branch runs:
nullspace functions run repo-agent --route /review --input-json '{"repo":"acme/api"}'
nullspace functions spawn repo-agent --route /report --input-json '{"repo":"acme/api"}'
nullspace functions schedule create repo-agent --cron "0 8 * * 1" --route /report
nullspace functions webhook create repo-agent --route /review --signing-secret "$WEBHOOK_SECRET"
import nullspace

fn = nullspace.Function.from_name("repo-agent")
review = fn.run(route="/review", input={"repo": "acme/api"})
calls = fn.spawn_map(
    [{"repo": "acme/api"}, {"repo": "acme/web"}],
    route="/report",
)
import { NullspaceClient } from "@nullspace/sdk";

const client = new NullspaceClient();
const fn = client.functions.get("repo-agent");

const review = await fn.run({
  route: "/review",
  input: { repo: "acme/api" },
});
const reports = await fn.spawnMap(
  [{ repo: "acme/api" }, { repo: "acme/web" }],
  { route: "/report" },
);

TypeScript Projects

TypeScript Functions use the same function.py descriptor. Use package.json scripts for the Node build lifecycle:
  • Jobs build during install and run the compiled artifact with entrypoint="node dist/index.js".
  • Services use scripts.start and entrypoint="npm run start" with a declared nullspace.Service(port=...).
  • nullspace dev can use tsx src/index.ts for local iteration; production deploys should not depend on tsx.
function.py
import nullspace

function = nullspace.Function(
    "typescript-minimal-job",
    mode="job",
    install="npm install && npm run build",
    entrypoint="node dist/index.js",
    outputs=nullspace.Outputs(paths=["result.json", "reports"]),
    max_concurrency=2,
)
nullspace functions init detects an existing Node package and can scaffold a minimal TypeScript job with src/index.ts, tsconfig.json, package.json build/dev scripts, and this shared descriptor. Invoke a deployed TypeScript Function with @nullspace/sdk:
import { NullspaceClient } from "@nullspace/sdk";

const client = new NullspaceClient();
const fn = client.functions.get("typescript-minimal-job", {
  environment: "staging",
});

const run = await fn.run({ input: { subject: "functions" } });
console.log((await run.wait()).status, await run.outputs());

const calls = await fn.spawnMap([
  { subject: "api" },
  { subject: "web" },
]);
console.log(calls.map((call) => call.id));
Use client.functions.call("fnrun_...") to rehydrate a detached call when only the persisted run/call ID is available. FunctionRun.wait(), logs(), outputs(), and cancel() operate on the backing run handle.

Templates And Warmup

Use a named template ref for simple functions, or provide a Template builder when the Function needs a custom machine environment. Deploy publishes the builder through the existing template pipeline and stores the built canonical template ref on the Function version.
function.py
import nullspace

template = (
    nullspace.Template.debian_slim(python="3.12")
    .apt_install("git")
    .pip_install("pandas", "claude-agent-sdk")
    .env({"HF_HUB_ENABLE_HF_TRANSFER": "1"})
    .workdir("/workspace")
)

function = nullspace.Function(
    "repo-review-agent",
    mode="job",
    template=template,
    workdir="/workspace/project",
    install="python -m pip install -e .",
    entrypoint="python main.py",
    warmup=nullspace.Warmup(
        command="python -c 'import pandas, claude_agent_sdk'",
        timeout_seconds=60,
    ),
)
The Function warmup hook runs inside the guest during deploy after install and before the version is finalized. Named Function secrets are not injected into warmup commands, so keep secret-dependent downloads in runtime code or a reviewed non-secret build path.

Run A Job

Use job when the agent has finite work: code review, report generation, benchmarking, data extraction, evaluation, or a one-shot framework run.
nullspace run repo-review-agent \
  --input-json '{"repo":"https://github.com/acme/repo"}' \
  --env OPENAI_API_KEY="$OPENAI_API_KEY" \
  --json

nullspace functions spawn repo-review-agent \
  --input-json '{"repo":"https://github.com/acme/repo"}' \
  --env OPENAI_API_KEY="$OPENAI_API_KEY" \
  --json

nullspace functions logs repo-review-agent --run fnrun_123
nullspace functions outputs repo-review-agent --run fnrun_123
nullspace functions status repo-review-agent --run fnrun_123
nullspace functions machine repo-review-agent --run fnrun_123
run invokes the Function and waits briefly for a terminal result. Long work returns a detached call with the backing run id. Use spawn when you always want the detached handle immediately. Set max_concurrency in function.py to cap concurrent job runs for a Function. run and invoke fail fast with 429 capacity_exceeded plus a Retry-After hint when the cap is saturated. spawn returns a detached call; when admitted to the bounded backlog, the call includes queued_reason and retry_after_seconds.
import nullspace

function = nullspace.Function.from_name("repo-review-agent")
result = function.run(input={"repo": "https://github.com/acme/repo"})

if isinstance(result, nullspace.FunctionCall):
    run = result.get(timeout=60)
else:
    print(result.outputs)

call = function.spawn(input={"repo": "https://github.com/acme/repo"})
print(call.id)

runs = function.map(
    [
        {"repo": "https://github.com/acme/api"},
        {"repo": "https://github.com/acme/web"},
    ],
    wait=120,
)
print([run.status for run in runs])
Use --version-id or SDK version_id= when you need to run a specific ready version instead of the active head. Use --idempotency-key or SDK idempotency_key= when retrying a create request after a client-side timeout.

Schedules And Inbound Webhooks

Schedules and inbound Function webhooks are thin triggers: each fire creates a normal Function run through the same invoke path and max_concurrency admission rules. They only target deployed Functions. Add schedules in function.py:
function.py
import nullspace

function = nullspace.Function(
    "repo-review-agent",
    mode="job",
    entrypoint="python main.py",
    schedules=[
        nullspace.Cron("0 8 * * 1", timezone="America/New_York"),
        nullspace.Period(seconds=21600),
    ],
)
Cron is wall-clock stable across redeploys. Period starts from the latest deploy, so use Cron when the real-world firing time matters. Operate schedules from the CLI or SDK:
nullspace functions schedule list repo-review-agent --json
nullspace functions schedule create repo-review-agent \
  --cron "0 8 * * 1" \
  --timezone America/New_York \
  --route /report \
  --input-json '{"source":"cron"}' \
  --json
nullspace functions schedule run repo-review-agent fnsch_123 --json
nullspace functions schedule fires repo-review-agent --trigger-id fnsch_123 --json
import nullspace

fn = nullspace.Function.from_name("repo-review-agent")
schedule = fn.create_schedule(
    kind="cron",
    expression="0 8 * * 1",
    timezone="America/New_York",
    route="/report",
    input={"source": "cron"},
)
fire = fn.run_schedule(schedule.id)
print(fire.call.id)
Create an inbound Function webhook with a write-only signing secret:
nullspace functions webhook create repo-review-agent \
  --signing-secret "$WEBHOOK_SECRET" \
  --route /review \
  --description github \
  --json

SIGNATURE="$(nullspace functions webhook signature "$WEBHOOK_SECRET" \
  '{"event":"push"}')"

curl -X POST "${NULLSPACE_API_URL}/v1/function-webhooks/fnwh_123" \
  -H "X-Nullspace-Signature: ${SIGNATURE}" \
  -H "Content-Type: application/json" \
  -d '{"event":"push"}'
Webhook payloads arrive at the Function as input["webhook"]["payload"]. Lifecycle webhooks under nullspace lifecycle webhooks ... are outbound machine lifecycle subscriptions; they are separate from inbound Function webhook triggers. The CLI covers the common trigger lifecycle: create, list, manual fire, and fire history. The API reference also exposes get, update, and delete operations for individual schedule and webhook trigger records.

Start A Service

Use service when the agent exposes HTTP, WebSocket, or another long-running server: an API wrapper around a graph, a web preview service, a chat backend, or a custom coordinator.
nullspace functions url repo-review-service \
  --env OPENAI_API_KEY="$OPENAI_API_KEY"

nullspace functions status repo-review-service --service fnsvc_123
nullspace functions logs repo-review-service --service fnsvc_123
nullspace functions restart repo-review-service
nullspace functions stop repo-review-service
url starts the latest active service when no service instance exists. Pass --service fnsvc_123 to resolve an existing service instance, and pass runtime env values only when a new service is being started.

Secure Endpoint Traffic

Set requires_proxy_auth=True when a public Function service endpoint should require Nullspace edge credentials before any request reaches the Function process:
function.py
function = nullspace.Function(
    "repo-review-service",
    mode="service",
    entrypoint="python -m uvicorn src.server:app --host 0.0.0.0 --port 8000",
    service=nullspace.Service(port=8000),
    policy=nullspace.Policy(public_url=True),
    requires_proxy_auth=True,
)
Mint a proxy-auth token, store the returned secret, and send the key and secret on endpoint traffic. The secret is returned only once.
nullspace functions proxy-auth create repo-review-service \
  --env prod \
  --name client-a \
  --route /invoke \
  --json

curl -H "Nullspace-Key: $NULLSPACE_FUNCTION_PROXY_AUTH_KEY" \
  -H "Nullspace-Secret: $NULLSPACE_FUNCTION_PROXY_AUTH_SECRET" \
  https://prod--repo-review-service.<public-host>/invoke

curl -H "Nullspace-Key: $NULLSPACE_FUNCTION_PROXY_AUTH_KEY" \
  -H "Nullspace-Secret: $NULLSPACE_FUNCTION_PROXY_AUTH_SECRET" \
  https://<public-host>/__ns-edge/functions/prod/repo-review-service/invoke
Use the host-style URL for DNS-safe environment and Function names. Use the path-style URL when a name contains characters that are not valid in DNS labels. Nullspace-Key and Nullspace-Secret are stripped before proxying. Missing, revoked, expired, wrong-route, or wrong-secret credentials return 401 at the edge. Operate tokens with the same Function reference and environment selector:
nullspace functions proxy-auth list repo-review-service --env prod --json
nullspace functions proxy-auth revoke repo-review-service fnpauth_123 --env prod --json
For app-controlled auth, enforce your own bearer token inside the service instead. Store the expected value as a named secret, check the incoming Authorization: Bearer ... header in your FastAPI, Express, or other app server, and leave Nullspace proxy-auth out of that decision path.

Operate A Function

Keep the function name stable. Deploy new bundles from the same project, then use run or service IDs for individual executions.
NeedCommand
List Functionsnullspace functions list --env prod --json
Inspect a Functionnullspace functions get <name> --env prod --json
View version historynullspace functions history <name> --env prod --json
Roll back to a versionnullspace functions rollback <name> --version-id fnv_123
Inspect raw event historyOpenAPI GET /v1/functions/{function_id}/events
See logsnullspace functions logs <name> --run fnrun_123
Read output metadatanullspace functions outputs <name> --run fnrun_123
Check statusnullspace functions status <name> --run fnrun_123
Inspect backing machinenullspace functions machine <name> --run fnrun_123
Run a debug commandnullspace shell <name> --run fnrun_123 --cmd "ls -la"
Cancel a job runnullspace functions stop <name> --run fnrun_123
Stop a servicenullspace functions stop <name>
Retain a stopped service machinenullspace functions stop <name> --service fnsvc_123 --retain-machine
Restart a service after deploynullspace functions restart <name>
Mint endpoint proxy-authnullspace functions proxy-auth create <name>
List endpoint proxy-authnullspace functions proxy-auth list <name>
Revoke endpoint proxy-authnullspace functions proxy-auth revoke <name> <token_id>
List schedulesnullspace functions schedule list <name>
Inspect trigger firesnullspace functions schedule fires <name>
List inbound webhooksnullspace functions webhook list <name>
Delete a functionnullspace functions delete <name> --yes

Checkpoint, Branch, And Rollback

Function checkpoints capture the backing machine for a live run or service instance. Use them to branch experiments from captured machine state, inspect lineage, or roll back the active Function target.
nullspace functions checkpoint repo-review-agent \
  --run fnrun_123 \
  --label before-change

nullspace functions branch repo-review-agent \
  --run fnrun_123 \
  --checkpoint-id chk_123 \
  --count 3 \
  --input-json '{"prompt":"try another strategy"}' \
  --json

nullspace functions checkpoint repo-review-service \
  --service fnsvc_123 \
  --label service-before-change

nullspace functions branch repo-review-service \
  --service fnsvc_123 \
  --checkpoint-id chk_456 \
  --count 2

nullspace functions rollback repo-review-agent --checkpoint-id chk_123
nullspace functions rollback repo-review-agent --version-id fnv_123
nullspace functions tree repo-review-agent --json
These checkpoints are platform machine snapshots. Framework session, conversation, thread, graph, or flow state is still app-owned unless it is already present in the machine filesystem or a mounted volume at checkpoint time. Rollback changes which version future runs or services use. It does not mutate already-finished runs, and it does not move framework-owned state between sessions.

Logs, Outputs, And Machines

Function jobs and services expose backing machine IDs. Use normal machine tools to inspect files, run commands, open PTY sessions, create snapshots, or destroy retained debug machines.
nullspace functions logs repo-review-agent --run fnrun_123 --tail 100
nullspace functions outputs repo-review-agent --run fnrun_123 --json
nullspace shell repo-review-agent --run fnrun_123 --cmd "ls -la"
nullspace functions machine repo-review-agent --run fnrun_123
Failed jobs can retain their machine for debugging:
nullspace run repo-review-agent \
  --retain-on-failure \
  --retention-seconds 3600
Then attach with PTY, SSH, commands, file downloads, or the desktop viewer when the function used a desktop template. See Debug retained machines for the human debugging workflow.

Secrets And Env Vars

Declare env var names with nullspace.Env. For reusable credentials, create a named secret and attach it to the function:
nullspace secret create openai OPENAI_API_KEY="$OPENAI_API_KEY"
function.py
function = nullspace.Function(
    "repo-review-agent",
    entrypoint="python main.py",
    env=nullspace.Env(required=["OPENAI_API_KEY"]),
    secrets=[nullspace.Secret.from_name("openai")],
)
For one-off overrides, pass values at runtime with --env, --env-file, or SDK envs=.
nullspace run repo-review-agent \
  --env OPENAI_API_KEY="$OPENAI_API_KEY" \
  --env SERPER_API_KEY="$SERPER_API_KEY"
Runtime values are injected into the agent process. They are not stored in function config, run rows, service rows, or platform-owned event metadata. Function processes also receive NULLSPACE_RUN_ID, NULLSPACE_FUNCTION_ID, NULLSPACE_VERSION_ID, NULLSPACE_ENVIRONMENT, and NULLSPACE_MACHINE_ID. Application code can still print or write secrets, so treat logs, output files, and .env files as sensitive. Create inline named secrets from Python when a deploy workflow owns secret material, or prefer pre-created secrets for CI and production:
function.py
import nullspace

function = nullspace.Function(
    "repo-review-agent",
    entrypoint="python main.py",
    secrets=[
        nullspace.Secret.from_name("openai", environment="prod"),
        nullspace.Secret.from_dotenv(".env.prod", name="warehouse", environment="prod"),
    ],
)

State And Sessions

Framework session, thread, checkpoint, or flow state stays app-owned. Store durable state in a mounted volume, a retained or auto-resumed machine, or an external database. For example, keep LangGraph checkpoints in your checkpointer storage, keep Claude session files in a volume if they must survive cleanup, and let CrewAI flows own their state and output artifacts. Do not rely on the ephemeral machine filesystem for durable conversation state unless the machine is retained or the state is written to a mounted volume.

Cache And Templates

functions deploy reports function build cache state in human output and JSON. The default auto policy falls back to running install inside each job run or service start when no ready artifact exists. Use nullspace functions deploy --rebuild to build a fresh reusable function artifact during upload, or --no-cache to bypass cache lookup and artifact writes for one deploy. Use a custom template for heavy system dependencies or slow base setup. Inside a builder, force_build() / forceBuild() inserts a per-step cache boundary; template builds can also use whole-build cache bypass with skip_cache=True, skipCache: true, or nullspace template build --skip-cache.

Examples

Minimal Job

Deploy a small Python job and collect result.json plus a reports directory.

Minimal Service

Deploy a small Python HTTP service and fetch its public URL.

OpenAI Agents SDK Job

Deploy an OpenAI Agents SDK app as a finite job.

Claude Agent SDK Job

Deploy a Claude Agent SDK app as a finite job.

LangGraph Service

Deploy a LangGraph-backed HTTP service.

CrewAI Job

Deploy a CrewAI project as a bounded job.

Machine + Function Composition

Call a Function from a customer-created machine and inspect retained run machine state.
The Function Jobs and Function Services guides include Python, TypeScript, and CLI examples in the same workflow pages.