What happens when an LLM's answer has to become a Java object instead of just another String?

Generative AI has made it surprisingly easy to build applications that can read code, explain it, and even review it.

Send a diff to a large language model and ask:

Review this code and tell me what is wrong.

You will probably get something useful back.

But there is a problem.

You get text.

And text is perfectly fine when a human is going to read the response.

It becomes much less useful when another piece of software needs to make decisions based on it.

Suppose I want my CI pipeline to ask:

• Is there a security vulnerability?

• Which file contains it?

• What exact line is affected?

• Is this issue serious enough to block the merge?

• How confident is the reviewer?

• Should this pull request be approved or rejected?

Now a paragraph of AI-generated prose is not enough.

I need a contract.

More specifically, as a Java developer, I want this:

ReviewReport report = reviewer.review(diff);

if (report.verdict() == Verdict.REQUEST_CHANGES) {
    System.exit(1);
}

Not this:

String response = model.call(diff);

// Good luck figuring out what this means.
if (response.contains("critical")) {
    ...
}

That distinction became the foundation of a small project I built:

a type-safe AI code reviewer using Java, Spring Boot, and Spring AI.

The interesting part of the project is not that an LLM can review Java code.

We already know models can do that.

The interesting part is what happens when we stop treating the model like a text generator and start treating its response like data.

---

The Starting Point: The AI Returns a String

The simplest version of an AI code reviewer is straightforward.

Give the model a Git diff:

- if (currency != "NGN") {
+ if (currency != expectedCurrency) {

Ask it to review the change and receive something like:

There appears to be an issue with the currency comparison.
Java String values should be compared using equals() rather than !=.
This could cause valid transactions to be rejected.

For a human reviewer, that is useful.

For an application, however, several questions immediately appear.

What is the severity?

What category of problem is this?

Which line is affected?

Is there a suggested fix?

Should the build fail?

You could try to extract these things from the response.

Maybe regex:

if (response.contains("security")) {
    ...
}

Maybe ask the model to return JSON and manually parse it.

But now your application depends on assumptions about text produced by a probabilistic system.

That is exactly the kind of boundary where I want more Java, not less.

So instead of asking the model for text, I defined what a review actually means in my application.

---

Turning the AI Response into a Java Contract

The core domain starts with a Finding.

public record Finding(

        @NotBlank
        String file,

        @Min(1)
        int line,

        @NotNull
        Severity severity,

        @NotNull
        Category category,

        @NotBlank
        String title,

        @NotBlank
        String explanation,

        String suggestedFix,

        @DecimalMin("0.0")
        @DecimalMax("1.0")
        double confidence

) {}

Now a review finding is no longer some paragraph that I need to interpret.

It has structure.

A finding must tell me:

file
line
severity
category
title
explanation
suggestedFix
confidence

The severity is also not arbitrary text.

public enum Severity {
    BLOCKER,
    MAJOR,
    MINOR,
    NIT
}

Neither is the category:

public enum Category {
    CORRECTNESS,
    SECURITY,
    CONCURRENCY,
    PERFORMANCE,
    STYLE,
    TESTING
}

And the overall review is another record:

public record ReviewReport(

        @NotBlank
        String summary,

        @NotNull
        List<@Valid Finding> findings,

        @NotNull
        Verdict verdict

) {}

With:

public enum Verdict {
    APPROVE,
    COMMENT,
    REQUEST_CHANGES
}

This is where the architecture becomes interesting.

The Java records are not simply DTOs used after the AI responds.

They define the contract the AI is expected to satisfy.

---

Spring AI Can Turn That Record into the Model's Output Schema

With Spring AI, the actual review call becomes remarkably small:

public ReviewReport review(String diff) {
    return chat.prompt()
            .user(u -> u.text("Review this diff:\n\n{diff}")
                    .param("diff", UnifiedDiff.withLineNumbers(diff)))
            .call()
            .entity(ReviewReport.class);
}

The important line is:

.entity(ReviewReport.class)

Instead of returning:

String

I am asking Spring AI for:

ReviewReport

That changes the relationship between the application and the model.

Spring AI can derive the structured-output schema from the Java type and use that structure when requesting the model's response. It then maps the response back into the Java record.

So conceptually the flow becomes:

Java record
     ↓
structured output schema
     ↓
LLM
     ↓
structured response
     ↓
Java record

The application code never needs to say:

JSONObject json = ...
json.get("severity");
json.get("line");
json.get("confidence");

And it certainly does not need regex to understand what the model meant.

The type system has moved all the way to the AI boundary.

---

Describing the Contract to the Model

Types tell us the shape of the data.

But sometimes the model needs more semantic information.

For that I use descriptions on the record properties.

For example:

@JsonPropertyDescription(
    "Line number in the NEW version of the file. " +
    "Must be a line present in the diff."
)
int line

Or:

@JsonPropertyDescription(
    "How sure you are, 0.0 to 1.0. " +
    "Be honest: use below 0.5 when you are guessing."
)
double confidence

And:

@JsonPropertyDescription(
    "Every problem found. Empty list if the diff is genuinely fine -- " +
    "do not invent findings."
)
List<@Valid Finding> findings

This creates an interesting pattern.

The record serves several purposes simultaneously.

It is:

• the Java domain model,

• the structured output contract,

• part of the model's instructions,

• and the object consumed by the rest of the application.

That reduces one of the problems that appears quickly in AI applications: having one description in your prompt, another in your parser, and another in your Java model.

Here, the contract lives much closer to the code that actually consumes it.

---

But Type-Safe Does Not Mean Correct

At this point it is tempting to say:

Great. The model returned a ReviewReport, therefore the response is valid.

Not quite.

Consider this:

{
  "file": "PaymentService.java",
  "line": 412,
  "severity": "BLOCKER",
  "category": "CONCURRENCY",
  "confidence": 0.97
}

That could map perfectly into our Java record.

The JSON is valid.

The enum values are valid.

The confidence is between 0.0 and 1.0.

There is only one problem:

the diff might not even contain line 412.

This is an important distinction when building AI systems.

There are different kinds of correctness.

First, there is structural correctness:

Does the response have the shape my program expects?

Then there is constraint correctness:

Do the values satisfy the rules of my domain?

And finally there is grounding:

Are the model's claims actually supported by the data it was given?

Types solve the first problem.

They do not automatically solve the other two.

---

Adding Bean Validation

Because the model output becomes an ordinary Java object, we can use ordinary Java validation.

For example:

@NotBlank
String file

@Min(1)
int line

@DecimalMin("0.0")
@DecimalMax("1.0")
double confidence

And nested findings are validated using:

List<@Valid Finding> findings

After receiving the report:

ReviewReport report = review(diff);

Set<ConstraintViolation<ReviewReport>> violations =
        validator.validate(report);

Now we have a second boundary:

MODEL
  ↓
STRUCTURED OUTPUT
  ↓
JAVA TYPE
  ↓
BEAN VALIDATION
  ↓
ACCEPT / REJECT

This is useful because a model producing syntactically valid JSON does not necessarily mean it has produced acceptable application data.

---

Retry the Contract, Not Every Failure

Once validation exists, another question appears.

What happens if the model gets the contract wrong?

One option is to fail immediately.

Another is to retry.

But blindly retrying every exception is dangerous.

An invalid API key will not magically become valid on attempt three.

A network configuration problem will probably still exist two seconds later.

An unsupported request parameter is not something the model can correct.

So the reviewer distinguishes between a contract failure and an infrastructure failure.

The project checks the exception chain for Jackson failures:

private static boolean isContractFailure(Throwable t) {
    for (Throwable c = t; c != null; c = c.getCause()) {
        if (c instanceof JacksonException) {
            return true;
        }
    }
    return false;
}

If the model returned something that could not be bound to ReviewReport, another attempt may help.

If the API call itself failed for an unrelated reason, the application fails fast.

That gives us a much more deliberate retry strategy:

Did the model violate the contract?
             │
        ┌────┴────┐
       YES        NO
        │          │
      retry     fail fast

The principle here extends beyond AI.

Retry only when another attempt has a reasonable chance of changing the outcome.

---

Structural Validation Still Cannot Catch Hallucinations

Now we reach what I think is one of the most important parts of the project.

Imagine the model returns:

new Finding(
    "src/main/java/PaymentService.java",
    412,
    Severity.BLOCKER,
    Category.CONCURRENCY,
    ...
)

Bean Validation sees:

file       → non-empty ✓
line       → greater than 0 ✓
severity   → valid enum ✓
category   → valid enum ✓
confidence → between 0 and 1 ✓

Everything passes.

But if the diff only touched lines 20–50, line 412 is hallucinated.

So I added another layer:

grounding validation.

---

Grounding the Review Against the Actual Diff

The reviewer parses the unified diff and knows which files and new-file lines were actually touched.

Then every finding is checked:

for (Finding f : report.findings()) {
    if (diff.covers(f.file(), f.line())) {
        grounded.add(f);
    } else {
        ungrounded.add(f);
    }
}

This produces two groups:

public record Result(
        List<Finding> grounded,
        List<Finding> ungrounded
) {}

Now the system can distinguish:

"The model produced valid data"

from:

"The model produced valid data about something that actually exists."

Those are very different guarantees.

If the model claims:

PaymentService.java:46

and line 46 exists in the supplied diff, the finding is grounded.

If it claims:

PaymentService.java:412

and that line is nowhere in the diff, the finding can be rejected.

The project can even explain why:

return "line %d of '%s' is not among the lines this diff touches"
        .formatted(f.line(), f.file());

So the validation pipeline now looks more like:

                  LLM
                   │
                   ▼
          Structured Output
                   │
                   ▼
             Java Record
                   │
                   ▼
           Bean Validation
                   │
                   ▼
          Grounding Validation
                   │
                   ▼
             ReviewReport

That is much closer to something I would trust inside an automated workflow.

---

Give the Model Line Numbers Instead of Asking It to Count

There is another subtle issue with reviewing diffs.

LLMs are not something I want to rely on for manually counting source lines.

So the system annotates the diff with the new-file line numbers before sending it to the model.

The system prompt explicitly says:

Every line of the diff is prefixed with its line number in the new file.
Use those numbers verbatim.
Do not count lines yourself and do not adjust them.

This is a useful AI engineering principle:

If your application can deterministically compute something, compute it yourself.

Do not spend model intelligence on work your program can do exactly.

Java can determine diff line numbers.

Let Java do it.

The model should spend its effort reasoning about whether those lines contain a bug.

---

A Diff Is Only a Keyhole

There is another limitation.

Sometimes a code review question cannot be answered from the diff.

Imagine seeing:

cache.put(accountId, value);

Is that a concurrency bug?

Maybe.

But to answer properly, I might need to know:

• What type is cache?

• Is the containing class a singleton?

• Is the field accessed by multiple request threads?

• Is it modified somewhere else?

• Is synchronization happening outside the visible change?

The diff does not tell us.

A diff is a keyhole into a much larger codebase.

The easiest thing for a model to do here is guess.

A better approach is to let it ask questions.

---

Tool Calling: Let the Reviewer Inspect the Repository

The project exposes repository operations as Spring AI tools.

For example:

@Tool(
    description = "Read a source file from the repository " +
                  "so you can see the code around a diff."
)
public String readFile(
        @ToolParam(
            description = "Repository-relative path, " +
                          "e.g. src/main/java/dev/oddy/Foo.java"
        )
        String path
) throws IOException {
    ...
}

There is also:

@Tool(
    description = "List the Java source files in the repository, " +
                  "so you can find related classes."
)
public List<String> listJavaFiles()

And:

@Tool(
    description = "Search the repository for a symbol and return " +
                  "the files and lines that mention it."
)
public List<String> findUsages(String symbol)

The model can now investigate instead of inventing context.

Conceptually:

                    ┌───────────────┐
                    │      LLM      │
                    └───────┬───────┘
                            │
                "I need more context"
                            │
                            ▼
                    ┌───────────────┐
                    │  Java Tools   │
                    ├───────────────┤
                    │ readFile()    │
                    │ findUsages()  │
                    │ listJavaFiles │
                    └───────┬───────┘
                            │
                            ▼
                       Repository

And enabling the tools on the reviewer is small:

this.chat = builder
        .defaultSystem(SYSTEM)
        .defaultTools(repoTools)
        .build();

The output remains:

ReviewReport

So adding tools does not require abandoning the type-safe output contract.

---

The Method Signature Becomes a Tool Contract

There is something elegant about tool calling in Java.

Consider:

public List<String> findUsages(String symbol)

That signature already tells us quite a lot.

The tool:

• has a name,

• accepts a String,

• returns a List<String>.

Annotations add semantic descriptions:

@ToolParam(
    description = "The identifier to search for, " +
                  "e.g. a field or method name"
)
String symbol

Again, Java's type system becomes part of the boundary between deterministic application code and probabilistic model reasoning.

This is the same general idea we used for structured output.

For output:

Java type → model schema

For tools:

Java method → tool schema

The application becomes an interesting combination of two worlds:

DETERMINISTIC JAVA              PROBABILISTIC AI

records                  <----> structured output
enums                    <----> constrained choices
Bean Validation          <----> model-generated values
methods                  <----> tool calls
filesystem checks        <----> model reasoning
diff parser              <----> review analysis

Instead of asking the model to replace our application logic, we surround the model with application logic.

---

Typed Does Not Mean Trusted

Giving an AI model tools introduces a new security boundary.

Suppose the model calls:

readFile("../../../../etc/passwd")

The parameter is perfectly type-safe.

It is a String.

That does not mean it is safe.

The repository tool therefore resolves the path and checks that it remains inside the repository root:

private Path safeResolve(String path) {
    Path target = root.resolve(path).normalize();

    if (!target.startsWith(root)) {
        throw new IllegalArgumentException(
            "Path escapes the repository root: " + path
        );
    }

    return target;
}

This is an important rule for AI tool calling:

Type-safe does not mean trusted.

Tool inputs still cross a trust boundary.

The model should receive the minimum capability required to perform the task.

In this project it can:

read repository source
list Java files
search for symbol usages

It does not receive arbitrary shell access.

That distinction matters.

---

From ReviewReport to CI Gate

Once the AI output is a proper domain object, something that was awkward with prose becomes trivial.

The report contains a verdict:

public enum Verdict {
    APPROVE,
    COMMENT,
    REQUEST_CHANGES
}

And the report exposes:

public int exitCode() {
    return verdict == Verdict.REQUEST_CHANGES ? 1 : 0;
}

Now an AI code review can participate in CI using the oldest integration API in computing:

the process exit code.

APPROVE          → 0
COMMENT          → 0
REQUEST_CHANGES  → 1

A shell script does not need to understand natural language.

GitHub Actions does not need to parse paragraphs.

Jenkins does not need to regex the model's answer.

The AI system produces a domain decision and ordinary software infrastructure consumes it.

This is one of the biggest benefits of structured output.

The rest of your application stops caring that the data originally came from an LLM.

---

How Do You Test an AI Code Reviewer?

This is where things become especially interesting.

A common instinct is to write a test like:

assertEquals(
    "The SQL query is vulnerable to SQL injection...",
    modelResponse
);

That is extremely brittle.

A model can produce a completely correct review using different wording.

It can reorder findings.

It can explain the same bug differently.

So I do not want to test prose.

I want to test invariants.

For example:

ReviewReport report = reviewer.review(diff);

assertThat(report).isNotNull();
assertThat(report.verdict())
        .isEqualTo(Verdict.REQUEST_CHANGES);

Then test the contract:

assertThat(
    validator.validate(reviewer.review(diff))
).isEmpty();

Then test an important known vulnerability in the sample:

assertThat(report.findings())
        .filteredOn(f -> f.category() == Category.SECURITY)
        .isNotEmpty()
        .allSatisfy(f -> {
            assertThat(f.severity())
                    .isEqualTo(Severity.BLOCKER);

            assertThat(f.line())
                    .isEqualTo(46);
        });

And test hallucination handling:

Grounding.Result result =
        Grounding.check(reviewer.review(diff), diff);

assertThat(result.ungrounded())
        .hasSize(1)
        .allSatisfy(f ->
            assertThat(f.line()).isEqualTo(412)
        );

Notice what the tests do not care about.

They do not assert the exact sentence the model wrote.

They assert properties that matter to the system.

That gives us a much more useful testing philosophy:

Do not test that the model says the same words every time. Test that its output continues to satisfy your application's contract.

---

Testing Without Calling the LLM Every Time

There is still another problem.

If every test calls a live model:

• tests become slower,

• CI needs an API key,

• failures may depend on network availability,

• every test run costs money,

• model changes can introduce noise.

So the project includes a replay model.

public class ReplayChatModel implements ChatModel {
    ...
}

Instead of contacting the provider, it serves a response recorded earlier.

The important part is that the rest of the application does not change.

The replay still goes through:

recorded model response
        ↓
structured binding
        ↓
ReviewReport
        ↓
Bean Validation
        ↓
grounding
        ↓
tests

This allows fast deterministic contract tests without pretending the AI layer does not exist.

The live model tests can then be run separately when needed.

That gives us two useful test categories.

Deterministic CI tests

Use recorded responses.

Fast, repeatable, no API key.

Live integration tests

Use the real provider.

Slower and potentially variable, but useful for verifying that the actual model still satisfies the contract.

This is very similar to how we already test other external integrations.

We do not hit a payment switch for every unit test.

We test our deterministic boundaries locally and keep real integration tests where they provide value.

---

The Sample Bugs

To make the reviewer testable, the repository includes a deliberately broken PaymentService diff.

It contains known issues including:

currency != expectedCurrency

for incorrect String reference comparison.

A check-then-act operation against a regular HashMap in a singleton context.

A BigDecimal.equals(BigDecimal.ZERO) comparison where scale can affect equality.

And, most seriously, an account identifier concatenated directly into SQL.

The SQL injection is planted at a known line:

line 46

That gives the test suite something concrete to assert.

There is also a clean diff.

That is equally important.

An AI code reviewer that always finds a problem may look impressive in a demo, but it is not a useful reviewer.

A good reviewer must also be capable of saying:

APPROVE

with:

findings = List.of();

when there genuinely is nothing worth reporting.

Testing false positives is just as important as testing bug detection.

---

The Architecture in One Picture

At this point the application can be summarized like this:

                         Git Diff
                            │
                            ▼
                  ┌──────────────────┐
                  │ UnifiedDiff      │
                  │ + line numbers   │
                  └────────┬─────────┘
                           │
                           ▼
                  ┌──────────────────┐
                  │    ChatClient    │
                  │                  │
                  │  System Prompt   │
                  │  Repository Tools│
                  └────────┬─────────┘
                           │
                           ▼
                       AI Model
                           │
                    tool calls?
                      /        \
                    yes        no
                    /            \
                   ▼              │
            ┌─────────────┐       │
            │ RepoTools   │───────┘
            └─────────────┘
                           │
                           ▼
                 Structured Response
                           │
                           ▼
                  ┌────────────────┐
                  │ ReviewReport   │
                  │ Java record    │
                  └───────┬────────┘
                          │
                          ▼
                  Bean Validation
                          │
                          ▼
                 Grounding Check
                          │
                          ▼
                 Trusted Findings
                          │
                          ▼
                    CI Exit Code

The LLM is important.

But notice how much of the reliability comes from everything around the LLM.

That is the part I find most interesting.

---

AI Applications Still Need Software Engineering

It is easy to look at generative AI and assume that traditional software engineering becomes less important.

Building this project left me with almost the opposite conclusion.

Once an LLM becomes part of a real application, familiar engineering concerns return immediately:

Contracts.

What shape of data is allowed?

Validation.

Which values are acceptable?

Security.

What capabilities should tools have?

Error handling.

Which failures should be retried?

Testing.

Which invariants must always hold?

Determinism.

Which work should Java perform instead of the model?

Integration boundaries.

How does the rest of the system consume the result?

The model introduces probabilistic reasoning.

It does not remove the need for deterministic engineering around that reasoning.

---

A Useful Mental Model

I now think about this kind of AI integration in three layers.

Layer 1: Ask

Give the model enough context to reason about the problem.

diff
system instructions
repository context

Layer 2: Constrain

Define exactly what your application is willing to accept.

records
enums
schema
Bean Validation

Layer 3: Verify

Check claims that deterministic code can verify.

Does this file exist?
Was this line actually touched?
Is this path inside the repository?
Does the output satisfy our invariants?

In other words:

ASK → CONSTRAIN → VERIFY

Do not ask the LLM to provide guarantees that your program can enforce itself.

---

Final Thoughts

The first version of an AI feature often looks like this:

String answer = ai.ask(question);

There is nothing wrong with that.

For chat interfaces, explanations, brainstorming, and many human-facing workflows, a String may be exactly what you need.

But when the model becomes part of an application workflow, the requirements change.

If another piece of software needs to consume the answer, then the AI response starts looking less like prose and more like an API response.

That means we should start asking familiar questions.

What is the schema?

What values are allowed?

How is it validated?

What happens when it violates the contract?

Which claims can be verified?

What capabilities does it have?

How do we test it?

And what does failure mean to the rest of the system?

For this code reviewer, the answer was to make Java responsible for what Java is good at:

types
contracts
validation
security boundaries
deterministic checks
tests

and let the model focus on what it is good at:

reasoning about code
understanding context
explaining problems
suggesting fixes

That combination is much more interesting to me than simply calling an LLM and printing its response.

Because once this works:

ReviewReport report = reviewer.review(diff);

the AI is no longer sitting beside the application producing text.

It has become a typed participant inside the application.

And that is where Java meets Generative AI.

---

Technology Used

The accompanying project uses:

• Java 21

• Spring Boot

• Spring AI

• Spring AI ChatClient

• Structured output mapped to Java records

• Jakarta Bean Validation

• Spring AI tool calling

• JUnit 5

• AssertJ

• OpenAI as the configured model provider

• Recorded/replay model responses for deterministic testing

The complete example progresses from a naive string-based reviewer through structured output, validation, grounding, repository tools, replay testing, and finally a CI-compatible review verdict.