NOCTURNE: Open-Source AI for Smarter Security Assessments

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

What if small cybersecurity teams could spend less time coordinating tools and more time understanding security risks?

Penetration testing often involves switching between terminals, scanners, browsers, notes, and reporting tools. For independent security professionals and small teams, managing this workflow can be time-consuming.

We built NOCTURNE, an open-source, AI-assisted penetration testing orchestration platform that brings the different stages of an authorized security assessment into one operator-controlled workspace.

NOCTURNE uses an LLM-based multi-agent architecture to coordinate assessment tasks. Instead of manually managing every tool and collecting results separately, an operator can define the scope of an engagement, supervise AI-assisted workflows, review findings, and prepare structured reports.

Our target users are freelance penetration testers, small cybersecurity consultancies, and internal security teams.

These users may benefit from a tool that helps organize repetitive assessment steps, reduces context-switching, and consolidates reporting.

We see potential for a commercial offering through managed deployments, hosted subscriptions, or per-engagement pricing.

Rather than replacing experienced security professionals, the goal is to help them work more efficiently while retaining human oversight.

This is an early commercial direction, not a claim that we already have paying customers.

The public demo allows visitors to explore NOCTURNE's operator interface directly in their browsers without installing dependencies or configuring API keys.

It includes sample sessions, agent activity, findings, and interactive dashboard components.

Important: The public deployment runs in simulation mode using mock data. It does not perform real penetration tests, invoke live AI models, or connect to an operational security backend. The live-execution architecture is available in the source repository for authorized environments.

The operator dashboard brings assessment activity, navigation, and monitoring into a unified interface.

Security findings can be organized and reviewed before being incorporated into reports.

The system also provides an interface for supervising browser-assisted assessment workflows.

GitHub Repository: https://github.com/NullClassifier/nocturne

NOCTURNE is publicly available under the MIT License.

The repository includes the application source code, development infrastructure, configuration examples, technical documentation, and instructions for local deployment.

Developers can inspect its architecture, experiment with supported model configurations, and contribute improvements.

NOCTURNE combines open-source AI orchestration frameworks, a modular application architecture, and security testing infrastructure.

At the center of the platform is LangGraph, which coordinates the agent workflow.

An orchestrator interprets the assessment objectives and delegates work to executor agents. Those agents can interact with available security tools, collect results, and update the engagement state.

The architecture is designed to keep the operator involved throughout the workflow, including the ability to steer or stop a run.

One of our main goals was to avoid locking the platform into a single proprietary AI provider.

We integrated LiteLLM to provide a configurable abstraction over different language-model endpoints.

NOCTURNE supports Gemma through compatible providers, including Google AI Studio, OpenRouter, and custom model-serving endpoints.

Our repository documents the use of a self-hosted Gemma model during development. This configuration allows model inference to run on infrastructure controlled by the operator rather than requiring a single external API.

Supporting different deployment options also gives users more flexibility over model costs, infrastructure, and data handling.

The backend is built with Python and FastAPI.

We separated the API from the agent execution process:

The engine also includes checkpointing capabilities so interrupted workflows can resume.

The interface is built with React, TypeScript, and Vite.

We wanted the experience to feel like a security operations workspace rather than a simple chatbot.

The interface combines live activity displays, agent visualization, findings management, terminal components, and report access.

For the public demonstration, we use the application's mock API implementation. This provides an interactive frontend without exposing real security testing infrastructure to the Internet.

NOCTURNE builds upon an earlier prototype, so not every component of the repository was developed during October 12–18.

During the challenge period, our documented improvements included:

These changes build on the earlier architecture and move NOCTURNE toward a more flexible and accessible developer experience.

The public Git history documents both the original foundation and the subsequent improvements.

For NOCTURNE, open innovation is not simply about publishing source code. It directly influences how the platform can be deployed, adapted, and maintained.

Different teams have different requirements for model quality, privacy, deployment, and cost.

Using open-weight models and an open-source model-integration layer makes it possible to experiment with alternatives instead of depending entirely on one proprietary provider.

2. Greater control over sensitive information

Security assessments can involve confidential technical information.

A self-hosted model deployment can provide greater control over where inference happens and how assessment information is processed, although secure deployment still depends on the surrounding infrastructure and configuration.

Usage-based API pricing can make recurring AI workloads harder to predict.

Running an open-weight model on infrastructure a team controls creates another option. It avoids per-token API fees, although hardware, hosting, and maintenance still have real costs.

NOCTURNE's model usage tracking also helps operators understand the cost of supported hosted inference.

Security workflows vary between organizations.

With an open-source architecture, developers can inspect agent orchestration, modify workflows, integrate additional tools, and improve the project's behavior.

This flexibility is especially valuable for specialized professional workflows where a generic AI assistant may not be sufficient.

Building an AI-assisted security platform involves more than connecting an LLM to tools.

We had to consider the boundaries between agent planning, execution, application state, and operator interaction.

Model compatibility is another challenge. Different providers may handle tool calls and structured responses differently, so reliable operation requires careful testing.

We also learned that a compelling interface is not a substitute for verified functionality. This is why our public demo is explicitly labeled as simulated, while the repository documents the separate live-execution architecture.

Before any commercial deployment, additional work on reliability, security hardening, access controls, and real-world validation would be essential.

We are entering the Best Use of Gemma category.

Gemma is an open-weight model supported by NOCTURNE's agent architecture. The project documents self-hosted Gemma usage and includes provider integrations designed to make compatible Gemma deployments easier to configure.

The motivation behind this integration is to make AI-assisted security workflows less dependent on proprietary models and fixed cloud providers.

Our interactive public demo is simulated; Gemma integration and model execution are documented in the source repository.

NOCTURNE was developed collaboratively by the Netrunners team.

We collaborated on project development, integration, and documentation, with some code contributions shared outside GitHub before being committed to the repository.

Our next steps are to improve compatibility across supported models, strengthen reliability, expand testing in authorized lab environments, and gather feedback from cybersecurity professionals.

We also want to explore whether a managed version could become a practical service for smaller security teams.

Our goal is to make AI-assisted security assessment workflows more accessible, adaptable, and transparent through open-source technology.

NOCTURNE is intended exclusively for educational research and authorized security assessments. Security testing must only be performed on systems owned by the operator or covered by explicit permission and an agreed testing scope.