AI-Built Software Security Audit

Section 1

Executive overview

1.1   Introduction

████████ Inc. (“the Client”), a Series-A B2B SaaS company, engaged Webisoft to perform a Vibe Check: an independent, human-led security and quality audit of a production web application their team assembled largely with an AI app builder. The application is roughly 4,200 lines of code and handles account sign-up, a customer dashboard, and card payments.

The review was delivered on the standard Vibe Check basis: a fixed fee of $250, read-only access to the source repository, and a signed report inside 24 hours. No live customer data, admin credentials, or passwords were requested or accessed at any point.

1.2   Audit summary

One senior engineer reviewed the full codebase by hand, supported by Webisoft’s automated first pass. The purpose of the audit was to answer, in plain English, three questions the Client’s customers and investors are starting to ask: is anything exposed, can one customer reach another’s data, and will the app hold up as it grows.

In summary, the engineer identified 3 Critical issues that warrant action before the next enterprise conversation, 5 Warnings worth scheduling into the coming sprints, and 12 checks that passed. The three Critical findings are common to AI-built applications: a live payment key shipped inside the app, an interface that returns any customer’s records when an ID is changed, and a password-reset flow that can be guessed. None require a rewrite; each has a concrete, costed fix in Section 6.

1.3   Test approach & methodology

A Vibe Check pairs an automated first pass with a full manual review, so that coverage is broad and every reported finding is verified by a human before it reaches the Client. The following phases were used:

• Automated first pass: secret scanning, dependency and CVE checks, and static analysis across the repository.
• Manual code review of authentication, access control, and payment logic, line by line.
• Data-flow tracing to confirm whether one account can reach another account’s data.
• Configuration review: CORS, error handling, environment and build output.
• Scalability read: query patterns, pagination, and behaviour under concurrent load.
• Human verification of every finding, with false positives discarded before delivery.

Risk methodology (Likelihood × Impact)

Each finding is rated by measuring the Likelihood that it will be exploited or hit in practice, and the Impact should that happen. Each axis runs 1 to 5. The two values combine into a severity band that maps to the Vibe Check scale used throughout this report.

Likelihood × Impact matrix

Placement of this engagement’s eight findings. Rows are Impact (5 at top), columns are Likelihood (5 at right).

1.4   Assessment scope

The assessment was scoped to the source code of the Client’s production web application, provided through a read-only repository invitation.

• Repository: ████████████ (read-only, single reviewer)
• Application: single-page front end, Node.js API, and a hosted database, generated and iterated with an AI app builder.
• Size: ~4,200 lines across front end, API, and configuration.
• In scope: application source, authentication and access control, payment integration, dependency manifest, build output, and deployment configuration.
• Out of scope: live customer data, third-party infrastructure internals, physical security, and social-engineering testing.

Access was limited to the single reviewing engineer and revoked on delivery of this report.

Section 2

Assessment summary & findings overview

Higher is healthier. The overall score is held down by the three Critical findings; each is individually fixable without a rewrite.

Findings overview

Status reflects a first-delivery report. On a live engagement, this column is updated as the Client remediates: Solved, Partially solved, Risk accepted, or Acknowledged.

Section 3

Findings & technical details

Every finding leads with what it means for your business, then the technical detail, redacted evidence, the fix, and the rough effort involved. Your $250 audit fee is credited against any of it.

// loaded client-side, visible in every browser
const STRIPE_SECRET = "sk-live-████████████████████";
const stripe = require("stripe")(STRIPE_SECRET);

app.get("/api/invoices/:id", auth, async (req, res) => {
  // returns the record for ANY id, no owner check
  const invoice = await db.invoices.findById(req.params.id);
  res.json(invoice); // GET /api/invoices/██ returns another tenant
});

// predictable token, no expiry stored
const token = Date.now().toString(36); // guessable
await db.resets.insert({ user: ████, token });
// reset accepted whenever token matches, forever

// req.query.q flows straight into the query object
const results = await db.users.find(
  { $where: `this.name == '${req.query.q}'` } // injectable
);

app.post("/api/login", async (req, res) => {
  // no rate limit, no lockout, unlimited attempts
  const ok = await checkPassword(req.body.email, req.body.password);
  res.json({ ok });
});

app.use(cors({ origin: "*" })); // any website may call the API

"dependencies": {
  "████████████": "^0.2.1", // unmaintained, CVE-20██-████
  "express": "^4.18.2"
}

app.use((err, req, res, next) => {
  // full stack trace returned to the client
  res.status(500).json({ error: err.stack }); // leaks internals
});

Section 4

Scalability read

const accounts = await db.accounts.findAll(); // no pagination: whole table
for (const a of accounts) {
  a.owner = await db.users.findById(a.ownerId); // N+1: one query per row
}
res.json(accounts);

Collapses around

1,000 concurrent users

Scalability score

64 / 100

Section 5

What passed

Twelve checks came back healthy. A Vibe Check reports the good with the bad, so you know what not to spend money on.

Section 6

Prioritized fix plan

In priority order. Each item shows rough effort. The fixes are written so your own AI or developer can execute the smaller ones; the deeper ones are worth an engineer.

Section 7

Closing & sign-off

William Marchand

Senior Engineer · Webisoft

Reviewed & signed · Vibe Check® · Report v1.0