@virtufin/workmanager

Virtufin WorkManager

Build Status NuGet PyPI npm

📖 Documentation: workmanager.docs.virtufin.com

Worker lifecycle management and code execution service with Dapr integration.

  • Worker lifecycle management (create, start, stop, delete)
  • Code execution engines: Python, C# Source, DotNet DLL
  • Pub/sub-based worker invocation via Dapr, with per-topic inboxes per worker
  • Sharded groups: several copies of a worker split a topic, one member per event
  • State store persistence for worker configurations and history
  • gRPC API with reflection
workmanager/
├── src/
│ ├── Virtufin.WorkManager/ # WorkManager: gRPC API, worker catalog, routing
│ ├── Virtufin.WorkManager.Runtime/ # Runtime: one Dapr actor per worker, hosts the engines
│ ├── Virtufin.WorkManager.Shared/ # WorkManager↔Runtime actor contract, shared host setup
│ ├── Virtufin.WorkManager.Protos/ # workmanager.proto + generated stubs (ship inside the Virtufin.WorkManager.Client NuGet)
│ ├── Virtufin.WorkManager.Client/ # IWorkManagerClient, WorkerSpec/CodeSource, IWorkerLoader, GatewayWorkManagerClient
│ ├── Virtufin.WorkManager.Engines.Local/ # in-process loaders (nupkg, C# source) for LocalWorkManager
│ ├── Virtufin.Worker.Host/ # Worker runtime contract (IWorker, IWorkerContext, WorkerEventRules, codec)
│ ├── Virtufin.WorkManager.Engines/ # Shared engine code (subprocess frames, nupkg layout, registry)
│ ├── Virtufin.WorkManager.Engine.*/ # Engine implementations (Python, CSharpSource, DotNetDll, NativeDll)
│ ├── python/virtufin/workmanager/ # Python client library
│ └── typescript/src/ # TypeScript client library
├── tests/
│ ├── python/ # Python client tests
│ ├── typescript/ # TypeScript client tests
│ └── Virtufin.WorkManager.*.Tests/ # .NET tests
├── docs/ # MkDocs documentation
├── deploy/ # Deployment configs
├── versions.env # Version pin
└── AGENTS.md # Agent documentation

All deployment infrastructure is managed in dedicated repositories.

Runs the whole backend (API, WorkManager, Runtime) natively with Dapr sidecars, built from the sibling source checkouts and configured from the helm chart. Only Redis runs in Docker. Requires the Dapr CLI, the .NET SDK, Docker and helm.

git clone https://git.haenerconsulting.com/virtufin/deploy-local.git
cd deploy-local
./scripts/deploy.sh # everything
./scripts/deploy.sh --service workmanager # or one service

See deploy-local for full documentation.

Deploy with the Virtufin Helm chart. Requires Dapr installed on the cluster.

helm repo add virtufin https://helm.haenerconsulting.com/virtufin/helm
helm install virtufin virtufin/virtufin \
--namespace virtufin \
--create-namespace \
--set stateStore.host=<redis-host> \
--set pubsub.host=<redis-host>

See helm for all configurable values.

WorkManager scales horizontally across different topics, not across the same topic. Multiple WorkManager replicas deployed against the same Dapr pubsub component will compete for messages on each topic, not share load — a message won by replica A is dropped at replica B (no re-routing). For throughput scaling, run multiple workers on different topics on the same replica, or optimize the worker code to handle higher per-message rates. See the Worker Manager Horizontal Scalability spec for the full design rationale.

Protocol Port Description
HTTP 25001 Health checks, management
gRPC 25002 Worker lifecycle operations
  • .NET 10.0 SDK or later
  • Dapr CLI (for local development)
dotnet build
dotnet test

Proprietary - Virtufin