byop-engine/README.md

Workflow Build & Push

sequenceDiagram
    participant Client as Client/API User
    participant APIHandler as API Handler (apps.go)
    participant BuildOrchestrationSvc as BuildOrchestrationService
    participant DB as Database (dbstore)
    participant BuildKitClientSvc as BuildKitClient (BuildMachineClient)
    participant GitRepo as Git Repository
    participant RegistryClientSvc as RegistryClient
    participant DockerRegistry as Docker Registry

    Client->>+APIHandler: POST /apps/{id}/build (Request Build)
    APIHandler->>+BuildOrchestrationSvc: RequestBuild(appId, buildConfig)
    BuildOrchestrationSvc->>+DB: CreateBuildJob(appId, status: PENDING)
    DB-->>-BuildOrchestrationSvc: buildJobId
    Note over BuildOrchestrationSvc: Enqueues buildJobId
    BuildOrchestrationSvc-->>-APIHandler: Ack (Build Queued, buildJobId)
    APIHandler-->>-Client: HTTP 202 Accepted (Build Queued)

    loop Process Build Queue
        BuildOrchestrationSvc->>BuildOrchestrationSvc: Dequeue buildJobId
        BuildOrchestrationSvc->>+DB: UpdateBuildJobStatus(buildJobId, FETCHING)
        DB-->>-BuildOrchestrationSvc: OK

        BuildOrchestrationSvc->>+BuildKitClientSvc: FetchSource(gitURL, commit, buildJobId)
        BuildKitClientSvc->>+GitRepo: Clone/Pull code
        GitRepo-->>-BuildKitClientSvc: Source code
        BuildKitClientSvc-->>-BuildOrchestrationSvc: Fetch Success / Logs

        BuildOrchestrationSvc->>+DB: UpdateBuildJobStatus(buildJobId, BUILDING)
        DB-->>-BuildOrchestrationSvc: OK

        BuildOrchestrationSvc->>+BuildKitClientSvc: BuildImage(sourcePath, dockerfile, buildArgs, imageName, buildJobId)
        Note over BuildKitClientSvc: Interacts with BuildKit daemon
        BuildKitClientSvc-->>-BuildOrchestrationSvc: Build Success / Logs / ImageID

        alt Build Successful
            BuildOrchestrationSvc->>+DB: UpdateBuildJobStatus(buildJobId, PUSHING)
            DB-->>-BuildOrchestrationSvc: OK

            BuildOrchestrationSvc->>+RegistryClientSvc: PushImage(imageID, imageNameWithTag, authConfig)
            Note over RegistryClientSvc: Delegates to BuildKitClient for actual push
            RegistryClientSvc->>+BuildKitClientSvc: PushImageInternal(imageID, imageNameWithTag, authConfig)
            BuildKitClientSvc->>+DockerRegistry: Push image layers
            DockerRegistry-->>-BuildKitClientSvc: Push Acknowledged
            BuildKitClientSvc-->>-RegistryClientSvc: Push Success / Logs
            RegistryClientSvc-->>-BuildOrchestrationSvc: Push Success / Logs

            BuildOrchestrationSvc->>+DB: UpdateBuildJobStatus(buildJobId, SUCCESS, imageTag)
            DB-->>-BuildOrchestrationSvc: OK
            BuildOrchestrationSvc->>+DB: UpdateAppCurrentImage(appId, imageTag, imageURI)
            DB-->>-BuildOrchestrationSvc: OK
        else Build or Push Failed
            BuildOrchestrationSvc->>+DB: UpdateBuildJobStatus(buildJobId, FAILED, errorMessage)
            DB-->>-BuildOrchestrationSvc: OK
        end
    end

Preview Service Workflow

sequenceDiagram
    participant Client as Client/API User
    participant APIHandler as API Handler (preview.go)
    participant PreviewSvcManager as PreviewServiceManager
    participant DB as Database (dbstore)
    participant LocalPreviewSvc as LocalPreviewService
    participant RemotePreviewSvc as RemotePreviewService
    participant BuildKitClientSvc as BuildKitClient (via PreviewCommon)
    participant DockerDaemon as Local Docker Daemon
    participant CloudProvider as Cloud Provider (e.g., OVH)
    participant RemoteVPS as Remote VPS

    Client->>APIHandler: POST /previews (appId)
    APIHandler->>PreviewSvcManager: CreatePreview(appId)

    alt Use Local Preview (dev/testing)
        PreviewSvcManager->>LocalPreviewSvc: CreatePreview(appId)
        activate LocalPreviewSvc
        LocalPreviewSvc->>DB: GetAppByID(appId)
        DB-->>LocalPreviewSvc: AppDetails
        LocalPreviewSvc->>DB: CreatePreviewRecord(appId, status: BUILDING)
        DB-->>LocalPreviewSvc: previewId
        Note over LocalPreviewSvc: Async: buildAndDeployPreview(preview, app)
        LocalPreviewSvc-->>PreviewSvcManager: Preview (pending)
        deactivate LocalPreviewSvc
        PreviewSvcManager-->>APIHandler: Preview (pending)
        APIHandler-->>Client: HTTP 200 OK (Preview pending)

        LocalPreviewSvc->>LocalPreviewSvc: buildAndDeployPreview()
        activate LocalPreviewSvc
        LocalPreviewSvc->>DB: GetAppComponents(app)
        DB-->>LocalPreviewSvc: Components
        LocalPreviewSvc->>BuildKitClientSvc: BuildComponentImages(components)
        activate BuildKitClientSvc
        Note over BuildKitClientSvc: Interacts with BuildKit daemon
        BuildKitClientSvc-->>LocalPreviewSvc: ImageNames, BuildLogs
        deactivate BuildKitClientSvc
        LocalPreviewSvc->>DB: UpdatePreviewBuildLogs(previewId, buildLogs)
        DB-->>LocalPreviewSvc: OK
        LocalPreviewSvc->>DB: UpdatePreviewStatus(previewId, DEPLOYING)
        DB-->>LocalPreviewSvc: OK
        LocalPreviewSvc->>LocalPreviewSvc: GeneratePreviewURL(localTLD)
        LocalPreviewSvc->>DB: UpdatePreviewVPS(previewId, "byop.local", "127.0.0.1", previewURL)
        DB-->>LocalPreviewSvc: OK
        LocalPreviewSvc->>LocalPreviewSvc: GenerateDockerCompose(imageNames, app, previewURL)
        LocalPreviewSvc->>DockerDaemon: docker-compose up -d (Traefik for routing)
        activate DockerDaemon
        DockerDaemon-->>LocalPreviewSvc: Deployment Success/Logs
        deactivate DockerDaemon
        LocalPreviewSvc->>DB: UpdatePreviewDeployLogs(previewId, deployLogs)
        DB-->>LocalPreviewSvc: OK
        LocalPreviewSvc->>DB: UpdatePreviewStatus(previewId, RUNNING)
        DB-->>LocalPreviewSvc: OK
        LocalPreviewSvc->>DB: UpdateAppPreview(appId, previewId, previewURL)
        DB-->>LocalPreviewSvc: OK
        deactivate LocalPreviewSvc
    else Use Remote Preview (production)
        PreviewSvcManager->>RemotePreviewSvc: CreatePreview(appId)
        activate RemotePreviewSvc
        RemotePreviewSvc->>DB: GetAppByID(appId)
        DB-->>RemotePreviewSvc: AppDetails
        RemotePreviewSvc->>DB: CreatePreviewRecord(appId, status: BUILDING)
        DB-->>RemotePreviewSvc: previewId
        Note over RemotePreviewSvc: Async: buildAndDeployPreview(preview, app)
        RemotePreviewSvc-->>PreviewSvcManager: Preview (pending)
        deactivate RemotePreviewSvc
        PreviewSvcManager-->>APIHandler: Preview (pending)
        APIHandler-->>Client: HTTP 200 OK (Preview pending)

        RemotePreviewSvc->>RemotePreviewSvc: buildAndDeployPreview()
        activate RemotePreviewSvc
        RemotePreviewSvc->>DB: GetAppComponents(app)
        DB-->>RemotePreviewSvc: Components
        RemotePreviewSvc->>BuildKitClientSvc: BuildComponentImages(components)
        activate BuildKitClientSvc
        Note over BuildKitClientSvc: Interacts with BuildKit daemon
        BuildKitClientSvc-->>RemotePreviewSvc: ImageNames, BuildLogs
        deactivate BuildKitClientSvc
        RemotePreviewSvc->>DB: UpdatePreviewBuildLogs(previewId, buildLogs)
        DB-->>RemotePreviewSvc: OK
        RemotePreviewSvc->>DB: UpdatePreviewStatus(previewId, DEPLOYING)
        DB-->>RemotePreviewSvc: OK

        RemotePreviewSvc->>CloudProvider: FindAvailablePreviewVPS() / ProvisionVPS()
        activate CloudProvider
        CloudProvider-->>RemotePreviewSvc: VPSDetails (vpsId, ipAddress)
        deactivate CloudProvider
        RemotePreviewSvc->>RemotePreviewSvc: GeneratePreviewURL(remoteTLD)
        RemotePreviewSvc->>DB: UpdatePreviewVPS(previewId, vpsId, ipAddress, previewURL)
        DB-->>RemotePreviewSvc: OK

        RemotePreviewSvc->>RemotePreviewSvc: GenerateDockerCompose(imageNames, app, previewURL)
        loop For each Image
            RemotePreviewSvc->>DockerDaemon: docker save image (to .tar)
            activate DockerDaemon
            DockerDaemon-->>RemotePreviewSvc: image.tar
            deactivate DockerDaemon
            RemotePreviewSvc->>RemoteVPS: SCP image.tar to /tmp/
            activate RemoteVPS
            RemoteVPS-->>RemotePreviewSvc: Transfer OK
            RemotePreviewSvc->>RemoteVPS: SSH: docker load -i /tmp/image.tar
            RemoteVPS-->>RemotePreviewSvc: Load OK
            deactivate RemoteVPS
            RemotePreviewSvc->>RemotePreviewSvc: rm /tmp/image.tar (local)
        end
        RemotePreviewSvc->>RemoteVPS: SCP docker-compose.yml to /home/debian/preview-{id}/
        activate RemoteVPS
        RemoteVPS-->>RemotePreviewSvc: Transfer OK
        RemotePreviewSvc->>RemoteVPS: SSH: cd /home/debian/preview-{id}/ && docker-compose up -d
        RemoteVPS-->>RemotePreviewSvc: Deployment Success/Logs
        deactivate RemoteVPS
        RemotePreviewSvc->>DB: UpdatePreviewDeployLogs(previewId, deployLogs)
        DB-->>RemotePreviewSvc: OK
        RemotePreviewSvc->>DB: UpdatePreviewStatus(previewId, RUNNING)
        DB-->>RemotePreviewSvc: OK
        RemotePreviewSvc->>DB: UpdateAppPreview(appId, previewId, previewURL)
        DB-->>RemotePreviewSvc: OK
        deactivate RemotePreviewSvc
    end