Engine System
The WorkManager uses a pluggable engine architecture to support multiple programming languages. Each engine handles code execution for a specific MIME type.
IEngine Interface
All engines implement the IEngine interface:
public interface IEngine
{
Task LoadCodeAsync(byte[] code, string workerId, CancellationToken cancellationToken = default);
Task<IReadOnlyList<CloudEvent>> ProcessAsync(CloudEvent input, CancellationToken cancellationToken = default);
WorkerPackageDependencies? ReadPackageDependencies(byte[] code) => null;
}
| Method | Description |
|---|---|
LoadCodeAsync |
Loads the worker code into the engine. Called once at worker creation or when code is updated; workerId is what the worker sees as IWorkerContext.WorkerId. |
ProcessAsync |
Processes an incoming CloudEvent into zero or more response CloudEvents, published in order; an empty list publishes nothing. |
ReadPackageDependencies |
The Virtufin.* packages the code declares, for library auto-reload; null when the format has none. |
NpmDependencies |
After a load: the @virtufin/* npm packages the worker asked for and the versions npm installed, for library auto-reload; null for engines without npm and workers without such a package. |
PyPiDependencies |
After a load: the virtufin-* Python packages a wheel's Requires-Dist declares and every virtufin-* version uv installed, for library auto-reload; null for source workers, engines without pip, and wheels without a virtufin-* requirement. |
Start and stop
IEngine.StartAsync(IWorkerContext, CancellationToken stop) and
StopAsync(CancellationToken deadline) carry the worker lifecycle
(docs/v1/worker-lifecycle.md#the-worker-contract) into the engine; both
default to no-ops. DotNetDllEngine implements them: in process it calls the
worker's StartAsync/StopAsync directly; through the CoreCLR bridge it
sends the context as a WorkerStartPayload (config, secrets, the state keys)
plus a host callback vtable (DotNetHostVtable, ABI 2: publish, status,
secrets, free_buffer) through which the bridge-side context's publishes,
status reports and GetSecretsAsync calls come back to the host's context
(secrets returns the freshly resolved map as JSON in a host-allocated buffer
the bridge releases with free_buffer; on failure the buffer holds the error,
which names a reference at most), and
SignalStop/StopWorker entry points for the stop token and StopAsync.
Each StartWorker creates a fresh stop token (a stopped worker started again
never sees the earlier stop), and the engine forwards the worker's publishes
from the StartWorker call on (a worker may publish inside its own
StartAsync) until StopAsync begins; the Runtime's gate decides ownership.
The subprocess engines (CSharpSourceEngine, PythonEngine, TypeScriptEngine) send the context
as a start frame (EngineStartPayload JSON: worker id, config, secrets) and
wait for started under the engine's startup timeout (never the 30 s
message timeout: a start may open a connection and log in). One reader task
demultiplexes the socket: the worker's host calls (publish, status,
state, secrets frames, each with a call id and answered by reply) are
carried out against the host's context while the engine's own requests
(start, process, ping, stop) stay one at a time, so a worker may
publish from its own loop, inside its start or while a process is in
flight. stop carries a budget; the subprocess cancels the worker's stop
token first, then runs its stop for at most the budget and answers stopped.
The host's stop token closes the engine's side of the publish gate at once. A
subprocess that dies or hangs is replaced, and a started worker is started
again in the new one with the same context. A host publish without a route
(no publishtopic, no type) is dropped, as in process.
The native engine calls a v3 library's Start and Stop exports, and the
host vtable's publish and report_status reach the host's context
(native ABI v3); a v2 library gets
no lifecycle calls.
The worker context
Every engine hands the worker an IWorkerContext on each call
(IWorker.ProcessAsync(input, context)): the worker id, the Runtime's api
client (WorkerContexts.EnvironmentApi, from VIRTUFIN_API_HOST/_GRPC_PORT/
_KEY/_TLS), a logger (the Runtime's, the bridge's stderr, or the native log
callback) and the call's cancellation token. Nothing is ambient: a worker that
uses state, pubsub or triggers reads context.Api, which is why the same
worker also runs under a LocalWorkManager with an in-memory api
(Worker.Host).
Architecture
The WorkManager maps a worker's mime type to a WorkManager.Runtime actor type
(RuntimeActorTypeNames); each actor type owns one engine class, and each
worker's actor owns one engine instance for its activation.
flowchart TD
subgraph WM["WorkManager (catalog, routing)"]
C["text/x-csharp"] --> A1[CSharpSourceWorker]
D["application/x-dotnet-dll"] --> A2[DotnetDllWorker]
N["application/x-native-dll"] --> A3[NativeDllWorker]
P["text/x-python, application/x-python"] --> A4[PythonWorker]
T["text/x-typescript, application/x-npm"] --> A5[TypeScriptWorker]
end
A1 --> E1[CSharpSourceEngine]
A2 --> E2[DotNetDllEngine]
A3 --> E3[NativeDllEngine]
A4 --> E4[PythonEngine]
A5 --> E5[TypeScriptEngine]
Engine Resolution
When a worker is created:
- The WorkManager picks the Runtime actor type for the mime type (an
unsupported one is
INVALID_ARGUMENT) and callsCreateAsyncon the worker's actor - In the background,
LoadAsyncmakes the actor construct its engine and callLoadCodeAsync()with the fetched code - On
StartWorkerthe actor subscribes the worker's topics; every message reaches the engine through the actor'sProcessAsync
Each worker has its own engine instance; no state is shared between workers.
Built-in Engines
PythonEngine
MIME Types: text/x-python (raw source) and application/x-python (a wheel) -- one engine handles both, since they're the same execution mechanism differing only in how the entry file gets prepared. LoadCodeAsync detects which format it was given by sniffing the code bytes for a ZIP local-file-header signature (every non-empty wheel has one; Python source never does), rather than needing the mime type threaded through. Formerly two engine classes (PythonEngine/PythonWheelEngine) and two Runtime actor types; merged since neither was a genuinely distinct execution mechanism.
Executes Python code in a persistent sandboxed python3 subprocess. For raw source, code is written to a .py temp file at startup; for a wheel, see "Wheel format" below. The subprocess is started with a Unix domain socket (python3 --socket <path> --code <path>). The subprocess stays alive for the worker's lifetime, communicating over the Unix domain socket (not stdin/stdout) in the worker wire protocol's binary frames (see Python Engine Internals).
Data flow:
sequenceDiagram
autonumber
participant Sub as Pub/Sub
participant WM as WorkManager<br/>(PythonEngine)
participant Py as python3 subprocess
participant Code as Worker's Process()
Sub->>WM: input CloudEvent
WM->>WM: write worker code to .py temp file
WM->>Py: start python3 X.py --socket S
Py->>WM: ready frame (over the Unix domain socket)
Note over WM,Py: engine waits for ready. timeout triggers relaunch.
WM->>Py: process frame (protobuf CloudEvent)
Py->>Code: Process(ce_dict)
Code-->>Py: result dict, list of dicts, or None
Py->>WM: result frame (protobuf CloudEventBatch)
WM->>WM: decode batch → publish each, in order<br/>(empty → no reply)
WM->>Sub: reply CloudEvent(s)
Note over WM,Py: 10s health-check ping (empty heartbeat).<br/>Unresponsive subprocess is auto-restarted.
Code Requirements:
def Process(event):
# event is a dict with CloudEvent fields
# Return a dict representing the output CloudEvent, a list of them
# (published in order), or None
return {
"type": "com.example.output",
"source": "my-worker",
"data": { "result": "processed" }
}
Lifecycle (optional): a module-level Start(ctx) runs once per
activation before the first event, Stop() after the last.
import threading
_ctx = None
def _loop():
while not _ctx.stopping.is_set(): # set the moment the host stops or loses the worker
_ctx.publish({"type": "com.example.tick", "source": "urn:ticker", "data": "{}"})
_ctx.stopping.wait(1.0)
def Start(ctx):
global _ctx
_ctx = ctx # worker_id, config, secrets, state, ...
ctx.report_status("connected", ctx.config.get("venue"))
threading.Thread(target=_loop, daemon=True).start()
def Stop():
pass # release what Start acquired; ctx.publish now raises HostRefused
ctx has worker_id, config, secrets (the snapshot at start),
stopping (a threading.Event), publish(event_or_list) (routed like a
returned output, through the host's outbox and ownership gate),
report_status(state, detail=None), state.get(key) → (value, etag) or
None, state.save(key, value, etag=None), state.delete(key),
state.keys(), and get_secrets() (resolved now). A refused call raises
HostRefused (a RuntimeError), a stale etag StateConflict, a value that
contains a secret ValueError; both exception classes are globals of the
worker module. The module runs in its own namespace, never the bootstrap's.
CloudEvent Mapping:
The bootstrap decodes the protobuf CloudEvent into a dict shaped like the CloudEvents JSON format:
{
"id": "event-123",
"type": "com.example.input",
"source": "urn:source",
"specversion": "1.0",
"datacontenttype": "application/json",
"time": "2026-10-24T12:00:00.123Z",
"sequence": 7,
"data": "{\"key\": \"value\"}"
}
datais the event's text data as astr(JSON is not parsed; the Python devkit'sCommandWorkerdoes that); binary data arrives base64-encoded indata_base64instead.- Every other attribute is a top-level key, keeping its type: integers as
int, booleans asbool,timeas an RFC 3339 string. - A returned dict maps back the same way: a non-string
datais JSON-encoded,data_base64becomes binary data,int/boolvalues become typed attributes, and a missingidorsourcegets a default.
Error Handling:
- An exception in Process() is sent back as an error frame; the engine throws it as the event's error and the subprocess stays alive
- stderr output is captured as warning-level log entries (not treated as errors); stdout is discarded
- Execution timeout is 30 seconds by default
- The subprocess is health-checked every 10 seconds; if unresponsive, it is automatically restarted
Wheel format (application/x-python): the code file is a wheel (a .whl, which is just a zip with a .dist-info/ folder) instead of a bare .py file, allowing third-party pip dependencies (raw source only supports stdlib-only single-file scripts). The published wheel itself is tiny — just worker.py plus a Requires-Dist dependency manifest, no vendored third-party code. LoadCodeAsync resolves and installs those dependencies at worker-load time via uv pip install --target <dir>, which installs both the wheel's own worker.py and its resolved dependencies flat into the same directory — exactly the layout the bootstrap script expects on sys.path (worker.py sitting alongside its now-installed dependencies). Otherwise the subprocess lifecycle, Unix-socket frame protocol, and health-check/auto-restart behavior are identical to the raw-source format.
Layout (fixed naming convention for the entry point; everything else is whatever the wheel's own build step put there):
worker-1.0.0-py3-none-any.whl
├── worker.py # entry point; module-level def Process(cloud_event): ...
└── <name>-<version>.dist-info/
└── METADATA # Requires-Dist: <dep> lines declare third-party dependencies
Code Requirements: same Process(event) contract as PythonEngine (see above) -- by the time worker.py is exec'd, uv pip install --target has already resolved and installed its declared dependencies into the same flat directory, already on sys.path, so imports of those dependencies resolve normally.
No import-allowlist equivalent for wheels. The raw-source path's ValidatePythonImports/blocked-modules check assumes decodable UTF-8 source text; it doesn't apply to zip bytes, and isn't applied to a wheel's worker.py. Dependencies are expected to be vetted via their Requires-Dist declaration, not restricted at runtime -- a deliberate asymmetry kept as-is when the two formats were merged into one engine, not something the merge changed.
Dependencies are installed at worker-load time, not build time. This keeps the published wheel itself KB-scale (Gitea's PyPI package registry silently fails to index uploads above ~8-10MB, well under what a vendored grpcio/protobuf wheel would need) and means the cluster does need outbound access to a package index at load time. Two indexes are consulted:
- Public/default index — PYTHON_DEFAULT_INDEX_URL env var (default https://pypi.org/simple/), configurable, used for everything except virtufin-* packages.
- Private virtufin index — fixed at Gitea's PyPI feed, authenticated via VIRTUFIN_PYPI_USERNAME/VIRTUFIN_PYPI_API_KEY env vars (Basic auth, embedded in the index URL), used for virtufin-* packages (e.g. virtufin-workmanager).
Installs run with uv's DEFAULT first-index index strategy, with the private index in the
--extra-index-url slot: uv always ranks extra indexes ABOVE the default index and stops at the
first index containing a package, so virtufin-* names always resolve from the private index and
can never be shadowed by a public package of the same name (dependency-confusion protection). A
version the private index lacks fails the install closed instead of falling back to the public
index. (The earlier --index-strategy unsafe-best-match flag disabled exactly that protection —
under it, a public package shadowing a virtufin-* name with a higher version would win; removed.
Both indexes are NOT equally trusted: the default index is public pypi.) A fixed UV_CACHE_DIR
(/tmp/worker-sockets/uv-cache) means repeated loads of the same dependency versions within a pod's
lifetime hit cache instead of re-downloading. Install timeout: TimeoutConstants.PythonDependencyInstallTimeoutSeconds (60s). See general-workers' Hello/GetTime workers and their @common/scripts/{build_python_wheel,publish_worker_package}.py build/publish pipeline for the reference implementation.
Library updates. A wheel worker is reloaded when a virtufin-* package it
runs on is released, direct or transitive. At load the engine records the wheel's
own virtufin-* requirements and what uv pip install installed (the install
directory's *.dist-info, the worker's own distribution excluded). Every
LIBRARY_UPDATE_POLL_SECONDS the Runtime looks up each installed virtufin-*
package on the private index's PEP 503 simple pages (uncached, so a release
minutes old counts), picking the newest final version that satisfies the wheel's
PEP 440 specifier, if it declares one; a result other than what uv installed
reloads the worker (stop, a fresh uv pip install, start), unless its config says
autoreload=false. The index has no dependency metadata, so only uv sees a cap
set by another package: a reload that doesn't install the version it was for
isn't repeated for that version, and the next release is a new attempt. A capped
package therefore costs one reload per release, never a loop. Third-party packages
are never followed.
TypeScriptEngine
MIME Types: text/x-typescript (one ES module) and application/x-npm (an
npm pack tarball). One engine and one Runtime actor type (TypeScriptWorker)
handle both; LoadCodeAsync tells them apart by the gzip header, as the Python
engine tells a wheel from source.
Execution: one persistent node subprocess per worker running the engine's
embedded bridge (worker-bridge.mts), over the same Unix-socket frame protocol
as the Python and C# source engines (start/started, process/result,
stop/stopped, host calls publish/status/state/secrets answered by
reply, one demultiplexing reader on each side). Node runs TypeScript with its
built-in type stripping: no compiler, no build step, about 25 ms once per
load. So a worker uses erasable syntax only (tsc --erasableSyntaxOnly): no
enum, namespace or constructor parameter properties. Node's
--experimental-transform-types, which would allow them, was removed in Node
26, so the engine does not use it. The Runtime image ships Node 24 LTS (the
node:<version>-bookworm-slim binary and npm, about 140 MB unpacked, 45 MB
compressed, as layers of their own that stay the same until the Node version
changes).
Code format. A worker module exports any of
export async function Start(ctx: WorkerContext): Promise<void> { ... } // once per activation, before the first event
export async function Process(event: WorkerEvent): Promise<Outputs> { ... }
export async function Stop(): Promise<void> { ... } // ctx.stopping is already aborted
or default-exports a worker object (or a class, constructed once) with
start(ctx), process(event) and stop(), which is how the
@virtufin/worker-devkit bases (WorkerBase, CommandWorker) are deployed
(export default new MyWorker()). Every entry point may be async. event is
the dict form every subprocess worker sees (the conformance vectors'
convention): id, source, specversion, type and every other attribute as
a top-level key, text data in data, binary data base64 in data_base64,
timestamps as RFC 3339 text. Process returns an event, a list (published in
order) or nothing; a non-string data is JSON-encoded, a Uint8Array is binary.
ctx is the context of every engine: workerId, config, secrets,
stopping (an AbortSignal), publish(events), reportStatus(state, detail),
state.get/save/delete/keys and getSecrets(), all returning promises; a refused
call rejects with an error named HostRefused (StateConflict for a stale
etag). Worker logs (console.*) go to stderr, which the Runtime logs.
Dependencies install with npm at load, into the worker's own directory:
text/x-typescript: the module's bare import specifiers are its dependencies (import { WorkerBase } from "@virtufin/worker-devkit"installs@virtufin/worker-devkit), each at its latest version. Node built-ins and relative imports are not packages. To pin a third-party version, deploy an npm package instead.application/x-npm: the tarball'spackage/package.jsonmainis the worker module (.js,.mjs,.tsor.mts; TypeScript in the package itself is stripped like a module, TypeScript insidenode_modulesis not, so dependencies ship JavaScript). Itsdependenciesare installed, every@virtufin/*one at its latest version whatever range it names (workers always run the newest Virtufin libraries);devDependenciesandscriptsare dropped.
@virtufin/* resolve from the private Gitea npm registry with the Runtime's
VIRTUFIN_REGISTRY_TOKEN (handed to npm in its environment and referenced as
${VIRTUFIN_NPM_TOKEN} from a temporary npmrc, never written to disk), every
other package from NPM_DEFAULT_REGISTRY (default https://registry.npmjs.org/).
npm's scope mapping is absolute, so a public package can never shadow a
@virtufin/* name. npm runs with --ignore-scripts --omit=dev: packages that
build native addons at install are not supported. The Runtime's registry
credentials are removed from the worker subprocess's environment.
Library updates. Like NuGet workers, a TypeScript worker is reloaded when a
@virtufin/* package it uses is released. At load the engine records the
worker's own @virtufin/* dependencies and every @virtufin/* package npm
installed (node_modules, nested copies included). Every
LIBRARY_UPDATE_POLL_SECONDS the Runtime re-resolves that closure from the
registry's packuments (uncached, with the same token): a range picks like npm,
the latest tag when it satisfies the range, else the highest satisfying
version. A result other than what npm installed reloads the worker (stop, a
fresh npm install, start), unless its config says autoreload=false. Only
@virtufin/* packages are followed: a third-party release never reloads a
worker.
Lifecycle, ownership, redelivery are the Runtime's, as for every engine;
the TypeScript devkit's WorkerBase derives output ids and drops repeated
inputs like the C# and Python bases.
CSharpSourceEngine
MIME Type: text/x-csharp
Deploys a worker from C# source text, no nupkg needed: a notebook can send the code it
just wrote. The Runtime starts one CSharpSource.Bridge child process per worker (a
self-contained JIT process; the Runtime itself is NativeAOT and cannot load compiled code).
The bridge resolves the source's NuGet packages, compiles it with Roslyn and runs it.
Example (the greeter worker; the same file is the test fixture
tests/Virtufin.WorkManager.Tests/Fixtures/greeter.worker.cs):
#r "nuget: Virtufin.Worker.DevKit"
using System.Text.Json.Nodes;
using System.Threading.Tasks;
using CloudNative.CloudEvents;
using Virtufin.Worker.DevKit;
using Virtufin.Worker.Host;
// Answers {"name": "Ada"} with {"greeting": "Hello, Ada!"}
// (ce-type com.virtufin.greeter.response, topic wrk.greeter.response).
public sealed class Greeter() : WorkerBase<JsonNode, JsonObject>("greeter", new StrictJsonCodec())
{
protected override Task<JsonObject?> ProcessAsync(CloudEvent input, JsonNode payload, IWorkerContext context) =>
Task.FromResult<JsonObject?>(new JsonObject
{
["greeting"] = $"Hello, {payload["name"]?.GetValue<string>() ?? "world"}!",
});
}
Sent {"name": "Ada"}, it publishes {"greeting": "Hello, Ada!"} as
com.virtufin.greeter.response on wrk.greeter.response.
Data flow:
sequenceDiagram
autonumber
participant Sub as Pub/Sub
participant RT as Runtime<br/>(CSharpSourceEngine)
participant Br as CSharpSource.Bridge<br/>(child process)
participant Feed as NuGet feeds
participant W as Compiled worker
Note over RT,W: === LoadCodeAsync (once per worker) ===
RT->>Br: start, code file path
Br->>Feed: resolve #r "nuget: …" packages (+ transitive)
Feed-->>Br: package DLLs
Br->>Br: Roslyn compile (BCL + host-shared + package DLLs)
Br->>W: load into a WorkerLoadContext, find the IWorker
Br-->>RT: ready (or error: the compiler/resolver message)
Note over Sub,W: === Per message ===
Sub->>RT: input CloudEvent
RT->>Br: process frame (protobuf CloudEvent)
Br->>W: ProcessAsync
W-->>Br: outputs
Br-->>RT: result frame (protobuf CloudEventBatch)
Package directives. Lines at the top of the source, before any code (blank and //
lines may sit between them):
| Directive | Meaning |
|---|---|
#r "nuget: Virtufin.Worker.DevKit" |
the latest stable release |
#r "nuget: Some.Package, 1.2.3" |
a NuGet version range; a bare version is a minimum |
Virtufin.Strategy.DevKit / Some.Package 1.2.3 |
the same, in Verso's syntax (* = latest) |
They are resolved by the DLL engine's installer (NuGetDependencyInstaller), transitively,
from the same two feeds with the same credentials: Virtufin.* from the org feed at the
newest release in the range, everything else from DOTNET_DEFAULT_NUGET_SOURCE_URL
(default nuget.org) at the lowest version in the range (latest when there is no
version). Only NuGet references are supported; #r "some.dll" fails the load. The
directive lines are blanked before compiling, so compiler line numbers match your source.
References. The worker compiles against the .NET 10 reference assemblies,
CloudNative.CloudEvents, Virtufin.Worker.Host, and every resolved package DLL. There
are no implicit usings; write them out. Nullable annotations are enabled.
Host-shared assemblies. A package assembly the bridge ships itself (Virtufin.Worker.Host,
CloudNative.CloudEvents, Virtufin.Api.Client, Google.Protobuf, Virtufin.Naming, …)
is compiled against and loaded from the bridge's copy, not the package's, so the worker's
CloudEvent and IWorker are the host's types. This is the same WorkerLoadContext rule
DLL workers follow. Everything else (Worker.DevKit, Core, the domain DevKit packages) loads from the
feed.
Entry point. Exactly one non-abstract class implementing IWorker (directly or
through a devkit base such as WorkerBase<,>, BehaviorWorkerBase or
StrategyWorkerBase), with a public parameterless constructor. It runs as a DLL worker
does: every output is published in order. Two such classes fail the load; make
helpers abstract. Without one, the original form still works: a static or instance
method CloudEvent ProcessAsync(CloudEvent input) (single output, null for none).
Library updates. A source worker with package directives is tracked like a DLL worker
(LIBRARY_UPDATE_POLL_SECONDS): when a newer Virtufin.* release is in range, the
Runtime reloads it, which resolves the packages and compiles the source again.
Per-event rules. Around every call the Runtime applies the shared rules in
Virtufin.Worker.Host (WorkerEventRules), the same for every engine:
flowchart LR
IN[Input event] --> PREP["PrepareInput<br/>content type, config, secrets"]
PREP --> W["Your IWorker<br/>(in the bridge)"]
W -->|outputs| FIN["FinishOutputs<br/>id + source, secrets stripped"]
FIN --> ROUTE["Route<br/>publishtopic, else ce-type"]
ROUTE --> OUT[Pubsub]
W -.->|exception| DLQ["DeadLetter<br/>dlq.<topic>"]
What each rule does, and the conformance vectors that pin them
(conformance/worker-event-rules.v1.json, also in the Virtufin.Worker.Host package), are
in Architecture: one event.
Running workers locally. Before deploying, the same worker can run in-process: the
LocalWorkManager in Virtufin.Behavior.DevKit (worker-devkit 0.34.0) subscribes workers to
topics on a local bus and applies the same WorkerEventRules. See
Running workers locally in the
worker-devkit docs.
Errors.
- A malformed directive, an unresolvable package, a compile error (
worker.cs(line,col): error CS…) or a missing/ambiguous entry point fails the load (LOAD_FAILED, with the message). - An exception the worker throws fails that message only; the next one is processed.
- The load may take up to 90 s cold (60 s NuGet install + 30 s); later loads hit the
packages cache in
/tmp/worker-sockets/nuget-cache.
Limits. One source file. Package DLLs only (no native assets, no analyzers or source generators). The worker shares the bridge's version of every host-shared assembly listed above.
NativeDllEngine
MIME Type: application/x-native-dll
Loads per-architecture native shared libraries (.so / .dylib / .dll) via NativeLibrary.Load inside the AOT-compiled WorkManager.Runtime process that hosts the worker. Exchanges CloudEvents with the worker's Process export in the worker wire protocol (C ABI v2: a protobuf CloudEvent in, a protobuf CloudEventBatch out) and supports host callbacks (log, gateway_call, free_response, and under ABI v3 publish and report_status) on the VirtufinHost struct. A library declaring ABI v3 also exports Start and Stop, the worker lifecycle (native ABI v3).
Data flow:
sequenceDiagram
autonumber
participant Sub as Pub/Sub
participant WM as WorkManager<br/>(NativeDllEngine,<br/>AOT)
participant Lib as Native worker<br/>(lib*.so)
participant Base as "AotNative<W>"
participant Ctx as WorkerContext
Note over WM,Lib: === LoadCodeAsync (once per worker) ===
WM->>WM: extract zip to temp dir
WM->>WM: find runtimes/<rid>/native/lib*.so
WM->>Lib: NativeLibrary.Load(path)
WM->>Lib: GetExport("Process") → fn-ptr
WM->>Lib: GetExport("FreeResult") → fn-ptr
WM->>WM: build VirtufinHost vtable<br/>(log, gateway_call, free_response)
Note over WM,Lib: --- one-time, <10ms startup ---
Note over Sub,Lib: === Per-message ProcessAsync ===
Sub->>WM: input CloudEvent
WM->>WM: CloudEventProtoCodec.Encode(ce) → inBytes
WM->>WM: inBuf = Marshal.AllocHGlobal(inBytes)
WM->>Lib: Process(host, inBuf, inLen, &outBuf, &outLen)
Lib->>Base: ProcessStatic<T>(host, inBuf, inLen, ...)
Base->>Base: CloudEventProtoCodec.Decode(inBuf) → ce
Base->>Base: _instances.GetOrAdd(host) → worker
Base->>Ctx: worker.ProcessAsync(ce, context) (Api = EnvironmentApi, Logger = native log callback)
Ctx-->>Base: output CloudEvents (zero or more)
Base->>Base: CloudEventProtoCodec.EncodeBatch(outputs) → outBytes
Base->>WM: status 0, outBuf = AllocHGlobal(outBytes), outLen = outBytes.Length
WM->>WM: Marshal.Copy(outBuf, span)
WM->>Lib: FreeResult(outBuf)
WM->>WM: CloudEventProtoCodec.DecodeBatch(span)
WM->>Sub: publish each output, in order
Isolation caveat: the engine runs inside the Runtime process; an unrecoverable native error terminates that Runtime pod and every worker it hosts (the WorkManager itself is unaffected). See NativeDllEngine Workers for the full author guide, C ABI, and defensive-coding requirements.
Performance: low-latency (~50-200 µs/message) without a subprocess boundary. Suitable for trusted, pre-compiled native code.
Code requirements:
- Worker is a per-architecture shared library at runtimes/<rid>/native/lib<library>.so (or .dylib / .dll), packaged in a standard NuGet .nupkg.
- The worker's .nuspec declares virtufin* extension elements (virtufinAbiVersion, virtufinEntryPoint, virtufinFreeResult, virtufinLibrary) inside <metadata>.
- Two exported C functions: Process(const VirtufinHost*, const uint8_t* in, int32_t in_len, uint8_t** out, int32_t* out_len) and FreeResult(uint8_t*).
- See NativeDllEngine Workers for a full example, the vendored virtufin_worker_api_c.h and the header-only C codec virtufin_worker_api_pb.h.
Error handling:
- ABI mismatch (NotSupportedException).
- Missing / malformed .nuspec, invalid library basename, missing RID entry (InvalidWorkerNuspecException / UnsupportedArchitectureException).
- Process returns 1 (VIRTUFIN_STATUS_WORKER_ERROR) → engine throws InvalidOperationException with the worker's UTF-8 message; the event fails like any worker error.
- Process returns 2 or any other status, or a malformed batch → engine throws InvalidWorkerResponseException.
DotNetDllEngine (inside the Runtime process, default)
MIME Type: application/x-dotnet-dll
Default engine as of LIBRARY_VERSION 0.0.59. The engine is
AOT-compiled into the WorkManager.Runtime binary and embeds the .NET
runtime (CoreCLR) on first use via libhostfxr. Worker DLLs
are loaded into a per-worker, collectible
AssemblyLoadContext. Per-call latency: sub-microsecond (direct
in-process method dispatch, no socket, no JSON).
See In-Process DotNet DLL Workers for the full architecture, setup contract, and trade-offs.
Data flow:
sequenceDiagram
autonumber
participant Sub as Pub/Sub
participant WM as WorkManager<br/>(DotNetDllEngine,<br/>AOT + CoreCLR)
participant Ctx as AssemblyLoadContext
participant W as IWorker impl
Note over WM: === First LoadCodeAsync (one-time) ===
WM->>WM: HostFxrBootstrap<br/>ensure CoreCLR loaded
Note over WM,Ctx: === Per-worker LoadCodeAsync ===
WM->>WM: WorkerNupkgLoader.Extract
WM->>WM: NuGetDependencyInstaller<br/>resolve+download nuspec deps
WM->>Ctx: new AssemblyLoadContext
WM->>Ctx: LoadFromStream(worker.dll)
WM->>Ctx: scan for IWorker impl
WM->>W: Activator.CreateInstance
Note over Sub,W: === Per-message ProcessAsync ===
Sub->>WM: input CloudEvent
WM->>W: worker.ProcessAsync(ce, context)<br/>(direct call, <1 µs)
W-->>WM: output CloudEvents (0..n)
WM->>Sub: reply CloudEvent
Code Requirements: Same as before — workers implement
Virtufin.Worker.Host.IWorker (usually via Worker.DevKit's WorkerBase). The
worker DLL is published as a NuGet .nupkg with a
lib/<tfm>/<id>.dll entry and a <virtufinLibrary> extension
in the nuspec. See
DotNet DLL Workers for the full author
guide.
Dependencies are resolved at worker-load time, not vendored into the nupkg.
The published nupkg's lib/<tfm>/ contains only the worker's own DLL;
third-party dependencies are declared in the nuspec's <dependencies>
element (standard NuGet convention — populated from the worker's
PackageReferences at build time) and resolved+downloaded by
NuGetDependencyInstaller (Virtufin.WorkManager.Engine.DotNetDll.Bridge)
the first time the worker loads, using NuGet.Protocol/NuGet.Resolver
driven in-process (no dotnet SDK is available in the runtime image,
unlike a build machine, so this can't shell out to dotnet restore).
This is the .NET analog of PythonEngine's wheel-format load-time uv pip
install — see that engine's section above for the motivating size-limit
issue this design avoids. Two sources are consulted: a configurable
public source (DOTNET_DEFAULT_NUGET_SOURCE_URL, default nuget.org)
for everything, and the existing authenticated private virtufin Gitea
feed (reusing VIRTUFIN_REGISTRY_USER/VIRTUFIN_REGISTRY_TOKEN, the
same credentials already used to fetch the nupkg itself) for
Virtufin.* packages. Resolved packages are cached under a fixed
/tmp/worker-sockets/nuget-cache so repeated loads of the same
versions within a pod's lifetime don't re-download. Vendored sibling
DLLs (old-style nupkgs published before this change) still take
precedence on a filename conflict, so already-published nupkgs
continue to load unchanged.
Discovery: The engine scans the loaded assembly for the
first non-abstract, non-interface type implementing IWorker.
If no implementation is found, an InvalidOperationException
is thrown.
Virtufin.Worker.Host provides the IWorker contract, the worker context
(IWorkerContext), the AOT-native bridge (AotNative<W>) and the
CloudEventProtoCodec (the worker wire protocol); see
Worker.Host. Workers are authored with
Worker.DevKit.
The same loading pipeline runs in-process for notebooks and tests:
Virtufin.WorkManager.Engines.Local provides
DotNetDllWorkerLoader (nupkg + nuspec dependencies + WorkerLoadContext)
and CSharpSourceWorkerLoader (the bridge's WorkerCompiler, no child
process) behind the client's IWorkerLoader.
Adding an Engine
- Implement
IEngine(inVirtufin.WorkManager.Engines) in a newVirtufin.WorkManager.Engine.*project. - Add a
WorkerRuntimeActorBase<TEngine>subclass with its own[DaprActor]type name inVirtufin.WorkManager.Runtime/Actors. - Add the mime type to
WorkerMimeTypes(Virtufin.WorkManager.Client) and map it to that actor type inRuntimeActorTypeNames(Virtufin.WorkManager.Shared). - Optionally, an
IWorkerLoaderinVirtufin.WorkManager.Engines.Localso the format also runs under aLocalWorkManager.
Engine Lifecycle
Engines are created fresh for each worker's actor activation. This means:
- Engines should be stateless (or handle their own state initialization)
- Engine instances are not reused across workers
- When an actor reactivates (idle timeout, Runtime restart), a new engine is created and the code loaded again
Thread Safety
Engine instances are specific to a single worker and accessed sequentially (one message at a time). However:
PythonEngineandCSharpSourceEnginestop their subprocess on disposeDotNetDllEngineunloads the worker's load context on disposeNativeDllEngineimplementsIAsyncDisposable; dispose frees the loaded library and releases the host struct
Performance Characteristics
The figures below are estimates. Measured numbers per engine (engine overhead, pipeline latency and saturation), from the nightly benchmark runs, are at benchmarks.docs.virtufin.com.
| Engine | Startup | Per-Message | Throughput | Recommended For |
|---|---|---|---|---|
| CSharpSourceEngine | ~2s (Roslyn compilation) + package resolution (cold: network-bound) | < 1ms | High | High-throughput topics, complex logic |
| DotNetDllEngine | < 10ms | < 1ms | Highest | Pre-compiled .NET assemblies |
| PythonEngine (raw source) | ~500ms (subprocess start) | 1-5ms | Medium | Prototyping, scripting, stdlib-only single-file Python |
| PythonEngine (wheel) | ~500ms + uv pip install (cold cache: network-bound; warm cache: fast) |
1-5ms | Medium | Python workers needing third-party pip dependencies |
| TypeScriptEngine (module) | ~0.5 s (node start + type stripping; measured 464 ms, first load) + npm install when it imports packages (one @virtufin/* import measured 2.0 s cold, 0.75 s with a warm npm cache) |
~50 µs (measured p50 46 µs, p99 134 µs; Python 39/135 µs on the same machine) | Medium | TypeScript/JavaScript workers, npm ecosystem |
| NativeDllEngine | < 10ms (per-RID resolve + dlopen) | ~50-200 µs | High | Pre-compiled native libraries (C/C++/Rust/Zig) where latency matters and the worker is trusted |
Python Engine Internals
The Python engine uses a persistent subprocess that communicates via a Unix domain socket:
- The subprocess is started once at
LoadCodeand stays alive for the worker's lifetime - Code is written to temp files under
/tmp/worker-sockets/code/<id>/:code.py(the worker source) andcode.bootstrap.py(a thin bootstrap thatexec()s the code file and runs the socket frame loop) - A Unix domain socket at
/tmp/worker-sockets/<id>.sockis bound by the C# side; the Python subprocess connects to it as a client (--socket) - Engine and subprocess exchange binary frames (
SubprocessFrames): a 4-byte big-endian length (of type byte + payload), a 1-byte frame type, then the payload - Frame types:
ready(1, subprocess → engine after init),process(2, payload: a protobufCloudEvent),result(3, payload: a protobufCloudEventBatch, empty = no output),error(4, payload: a UTF-8 message),ping/pong(5/6, heartbeat),shutdown(7, graceful stop),start/started(8/9, payload:EngineStartPayloadJSON),stop/stopped(10/11, payload: the budget in ms, 4 bytes big-endian), and the worker's host callspublish(12, aCloudEventBatch),status(13),state(14),secrets(15), each prefixed by a 4-byte call id and answered byreply(16: the call id, a status byteok/refused/invalid/conflict/failed, the result) - A reader thread demultiplexes the socket (replies to the waiting call,
pinganswered at once, requests queued) and the main thread handlesstart/process/stopone at a time - The subprocess sends
readybefore entering the processing loop; the engine blocks until it receives it (or anerrorframe on init failure) - The CloudEvents protobuf codec is pure Python inside the bootstrap, so workers stay stdlib-only
- A health check pings the subprocess every 10 seconds with an empty
ping; if the subprocess fails to respond, it is automatically restarted on the nextProcessAsynccall - Stderr is captured as a Warning-level log channel, not treated as an error
- Stdout is discarded — all protocol communication goes over the Unix domain socket
Recommendations
- Use CSharpSourceEngine for high-throughput or latency-sensitive topics
- Use PythonEngine for rapid prototyping, scripting, or when Python-specific packages are needed
- Use TypeScriptEngine for TypeScript/JavaScript workers and the npm ecosystem; per-message cost is the Python engine's
- The first Roslyn compilation in CSharpSourceEngine incurs a ~2s startup cost; subsequent messages are sub-millisecond
- Python worker subprocess overhead is ~500ms at startup and 1-5ms per message