Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI summary #183

Merged
merged 7 commits into from
Sep 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 46 additions & 17 deletions apps/api/src/lib/rating.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ type GenerateRatingInput = {
fileExtension: "json" | "yaml";
};

/**
* @description produces a stripped down version of the report which can be fed
* to LLM models.
*/
const getReportMinified = (fullReport: RatingOutput) => {
const issues = fullReport.issues;
return issues
Expand Down Expand Up @@ -57,32 +61,53 @@ const getReportMinified = (fullReport: RatingOutput) => {
);
};

const getOpenAiSummary = async (issueSummary: object) => {
const getOpenAiResponse = async (
messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[],
) => {
const response = await openai.chat.completions.create({
model: "gpt-3.5-turbo",
messages: [
{
role: "system",
content:
"You are an expert in REST API Development and know everything about OpenAPI and what makes a good API. You like chatting in a playful, and a somewhat snarky manner. Don't make fun of the user though.",
},
{
role: "user",
content: `Here's a summary of issues found in an OpenAPI file. The format is a JSON, where the first level indicates the severity of the issue (the lower the key, the more severe), the second level is the name of the issue. These mostly match up to existing spectral rulesets, so you can infer what the issue is. The third level contains the number of occurrences of that issue.\n\nI would like a succinct summary of the issues and advice on how to fix them. Focus on the most common issues and the highest severity. Keep the tone casual and playful, and a bit snarky. Also, no bullet points. Maximum of 3 issues please. Rank by severity and then occurrences.\n\nHere's the issue summary\n ${JSON.stringify(
issueSummary,
)}`,
},
],
messages,
temperature: 0.5,
max_tokens: 400,
top_p: 1,
frequency_penalty: 0,
presence_penalty: 0,
});
console.log("response", response);
return response.choices[0].message.content;
};

const getOpenAiLongSummary = async (issueSummary: object) => {
return await getOpenAiResponse([
{
role: "system",
content:
"You are an expert in REST API Development and know everything about OpenAPI and what makes a good API. You like chatting in a playful, and a somewhat snarky manner. Don't make fun of the user though.",
},
{
role: "user",
content: `Here's a summary of issues found in an OpenAPI file. The format is a JSON, where the first level indicates the severity of the issue (the lower the key, the more severe), the second level is the name of the issue. These mostly match up to existing spectral rulesets, so you can infer what the issue is. The third level contains the number of occurrences of that issue.\n\nI would like a succinct summary of the issues and advice on how to fix them. Focus on the most common issues and the highest severity. Keep the tone casual and playful, and a bit snarky. Also, no bullet points. Maximum of 3 issues please. Rank by severity and then occurrences.\n\nHere's the issue summary\n ${JSON.stringify(
issueSummary,
)}`,
},
]);
};

const getOpenAiShortSummary = async (issueSummary: object) => {
return await getOpenAiResponse([
{
role: "system",
content:
"You are an expert in REST API Development and know everything about OpenAPI and what makes a good API. You like chatting in a playful, and a somewhat snarky manner. Don't make fun of the user though.",
},
{
role: "user",
content: `Here's a summary of issues found in an OpenAPI file. The format is a JSON, where the first level indicates the severity of the issue (the lower the key, the more severe), the second level is the name of the issue. These mostly match up to existing spectral rulesets, so you can infer what the issue is. The third level contains the number of occurrences of that issue.\n\nI would like a succinct summary of the issues in 2 lines. Keep the tone casual and playful, and a bit snarky. Do not insult the user or API creator or the API. Also, no bullet points. Only talk about the highest severity issues.\n\nHere's the issue summary\n ${JSON.stringify(
issueSummary,
)}`,
},
]);
};

export const uploadReport = async ({
reportId,
fullReport,
Expand Down Expand Up @@ -331,7 +356,10 @@ const getReport = async (
}

const issueSummary = getReportMinified(output);
const openAiSummary = await getOpenAiSummary(issueSummary);
const [openAiLongSummary, openAiShortSummary] = await Promise.all([
getOpenAiLongSummary(issueSummary),
getOpenAiShortSummary(issueSummary),
]);
const simpleReport = {
version:
// TODO: Clean this up
Expand All @@ -350,7 +378,8 @@ const getReport = async (
score: output.score,
securityScore: output.securityScore,
sdkGenerationScore: output.sdkGenerationScore,
summary: openAiSummary ?? undefined,
shortSummary: openAiShortSummary ?? undefined,
longSummary: openAiLongSummary ?? undefined,
};

return Ok({
Expand Down
9 changes: 8 additions & 1 deletion apps/web/src/app/report/[id]/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import ShareButton from "@/components/ShareButton";
import DynamicBackground from "@/components/DynamicBackground";
import { FullReport } from "./full-report";
import { SimpleReport, getSimpleReport } from "./simple-report-request";
import ReportSummary from "./report-summary";

const HeroScore = async ({ simpleReport }: { simpleReport: SimpleReport }) => {
return (
Expand Down Expand Up @@ -37,7 +38,13 @@ const ReportPage = async ({ params }: { params: { id: string } }) => {
return (
<>
<HeroScore simpleReport={simpleReport} />
{simpleReport.summary ? <div>{simpleReport.summary}</div> : null}
{simpleReport.shortSummary && simpleReport.longSummary ? (
<ReportSummary
shortSummary={simpleReport.shortSummary}
longSummary={simpleReport.longSummary}
score={simpleReport.score}
/>
) : null}
<FullReport reportId={params.id} fileExtension={fileExtension} />
</>
);
Expand Down
39 changes: 39 additions & 0 deletions apps/web/src/app/report/[id]/report-summary.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
"use client";

import getScoreTextColor from "@/utils/get-score-test-color";
import { useState } from "react";

type ReportSummaryProps = {
shortSummary: string;
longSummary: string;
score: number;
};

const ReportSummary = ({
shortSummary,
longSummary,
score,
}: ReportSummaryProps) => {
const [isExpanded, setIsExpanded] = useState(false);
const scoreTextColor = getScoreTextColor(score);
return (
<div className="my-10 flex flex-col overflow-hidden rounded-lg bg-white p-8 shadow-md md:p-10">
<h3
className={`mb-6 font-roboto-mono text-xl font-bold uppercase ${scoreTextColor}`}
>
Summary
</h3>
<p className="whitespace-pre-wrap text-base">
{isExpanded ? longSummary : shortSummary}
</p>
<button
className="mt-4 text-blue-500 hover:text-blue-700"
onClick={() => setIsExpanded(!isExpanded)}
>
{isExpanded ? "Show less" : "Show more"}
</button>
</div>
);
};

export default ReportSummary;
3 changes: 2 additions & 1 deletion apps/web/src/app/report/[id]/simple-report-request.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ export type SimpleReport = {
fileExtension: "json" | "yaml";
title: string;
version: string;
summary?: string;
shortSummary?: string;
longSummary?: string;
};

export const getSimpleReport = async (
Expand Down
2 changes: 1 addition & 1 deletion apps/web/src/components/DetailedScoreSection/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ const DetailedScoreSection = ({
<thead className="contents text-left font-bold uppercase">
<tr className="contents text-gray-400">
<th>Severity</th>
<th>Suggestion</th>
<th>Issue</th>
</tr>
</thead>
<tbody className="contents">
Expand Down