Zero-Cloud AI Code Review: Connecting Self-Hosted GitLab to Local CUDA LLM Runtimes

For enterprises operating under strict regulatory compliance (finance, healthcare, defense, or proprietary IP vaults), sending proprietary source code to third-party multi-tenant cloud APIs is an absolute dealbreaker.

Yet, engineering teams still demand the productivity acceleration of AI-assisted code reviews, automated security auditing, and inline suggestions.

Fortunately, modern open-weights foundation models combined with GitLab Self-Managed and local CUDA acceleration make it entirely possible to deploy a 100% air-gapped, zero-cloud AI code review pipeline.

In this architecture guide, I will walk you through:

In a zero-cloud setup, all data remains strictly within the local area network (LAN) or on-premise workstation cluster:

+-------------------------------------------------------------+ | Developer Machine (Local LAN) | | git push origin feature/security-hardening | +-------------------------------------------------------------+ | v +-------------------------------------------------------------+ | Self-Hosted GitLab (Docker / Omnibus) | | - Triggers Merge Request Webhook | +-------------------------------------------------------------+ | v +-------------------------------------------------------------+ | Local AI Review Worker (Workstation) | | - C++20 / Python Service running on RTX/CUDA | | - Low-latency KV-Cache (Paged Allocation) | | - Open-Weights Models (e.g., CodeLlama / DeepSeek-Coder) | +-------------------------------------------------------------+ | v (GitLab API: POST /notes) +-------------------------------------------------------------+ | Merge Request Inline Review Comments | | - Flags potential buffer overflows and SQL injections | | - Suggests memory optimizations | +-------------------------------------------------------------+ Enter fullscreen mode Exit fullscreen mode 2. Setting Up the Local GPU Inference Engine To achieve fast review cycles on local hardware (such as an NVIDIA RTX workstation), inference speed and KV-cache efficiency are paramount.

Using optimized C++20 and CUDA backends, your inference server exposes an OpenAI/GitLab-compatible /v1/chat/completions endpoint listening exclusively on 127.0.0.1:8000.

nvidia-smi --query-gpu = name,memory.total,utilization.gpu --format = csv Enter fullscreen mode Exit fullscreen mode name, memory.total [MiB], utilization.gpu [%] NVIDIA GeForce RTX 4090, 24564 MiB, 0 % Enter fullscreen mode Exit fullscreen mode With an optimized local inference runtime, time-to-first-token (TTFT) for a 4,000-token git diff is under 80ms, running entirely offline with zero network packets leaving the local machine.

We configure an on-premise GitLab Runner tagged with local-gpu to intercept merge request diffs, format a code-audit prompt, and send it to our local inference endpoint.

stages: - security_review ai_code_review: stage: security_review tags: - local-gpu rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" before_script: - apt-get update && apt-get install -y curl jq git script: - | # 1. Extract the git diff of the current Merge Request against the target branch git fetch origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME GIT_DIFF=$(git diff origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME..HEAD) if [ -z "$GIT_DIFF" ]; then echo "No diff found. Skipping review." exit 0 fi # 2.

Construct the security prompt PROMPT="You are a senior security architect reviewing a GitLab Merge Request. Analyze this diff for vulnerabilities, memory leaks, and logic errors. Provide concise, line-specific markdown recommendations:\n\n$GIT_DIFF" # 3.

Call local air-gapped LLM endpoint PAYLOAD=$(jq -n --arg p "$PROMPT" '{ model: "local-code-reviewer", messages: [{role: "user", content: $p}], temperature: 0.1, max_tokens: 1500 }') REVIEW_OUTPUT=$(curl -s -X POST http://127.0.0.1:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d "$PAYLOAD" | jq -r '.choices.message.content') # 4. Post the review as an MR Discussion Note via GitLab REST API COMMENT_PAYLOAD=$(jq -n --arg b "$REVIEW_OUTPUT" '{ body: ("🤖 **Local AI Security Audit (Air-Gapped Review):**\n\n" + $b) }') curl -s -X POST "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}/notes" \ -H "PRIVATE-TOKEN: ${GITLAB_BOT_PAT}" \ -H "Content-Type: application/json" \ -d "$COMMENT_PAYLOAD" Enter fullscreen mode Exit fullscreen mode 4. Why This Architecture Wins for Enterprise Security Absolute Zero Data Egress: Network policies block all outbound internet traffic on the runner node.

All tokens are computed strictly within on-premise GPU VRAM. Deterministic Compliance: Fully satisfies ISO 27001, HIPAA, and GDPR air-gapping requirements for intellectual property and patient records. No Per-Token Billing: Cloud API token costs scale linearly with team size; local workstations process unlimited reviews at zero marginal cost.

Instant Turnaround: Developers receive comprehensive security and code quality audits within seconds of clicking "Create Merge Request". 5. Integrating with GitLab Duo & The Future While this custom pipeline provides a standalone review bot, GitLab is aggressively expanding GitLab Duo Self-Hosted Models, allowing organizations to bring their own local model weights (vLLM, custom endpoints) directly into Duo Chat and Code Suggestions.

By mastering local inference runtimes today, your team is prepared to transition smoothly as GitLab standardizes on-premise LLM gateways.

Is your team exploring self-hosted or local AI models for source code review? Let’s exchange notes in the comments!