Review Comment Analytics in GitLights: Sentiment, Type, Urgency and Technical Depth

In most engineering teams, review comments are the hidden layer of collaboration. They contain design discussions, risk assessments, mentoring moments and implicit standards. Yet they are rarely analyzed beyond a simple count of how many comments exist on a pull request.

The GitLights Comments dashboard is built to change that. Instead of treating comments as raw text blobs, it applies natural language processing (NLP) and structured aggregation to expose:

• How review comments evolve over time, with EMA and RSI.
• Sentiment patterns across periods and as a global distribution.
• Comment type taxonomies (bug reports, questions, documentation, testing, etc.).
• Technical categories such as performance, security or architecture.
• Urgency levels (low, medium, high, critical).
• Derived indicators per developer and per repository: urgency, word counts, technical depth, sentiment, readability, constructiveness, questions, action items, impact score, % positive, and high‑priority counts.

This article walks through how the Comments dashboard works from a product and metrics perspective, and how engineering leaders can use it to reason about review quality, collaboration patterns, and the real impact of code discussions.

1. From raw review comments to structured analytics

GitLights ingests review comments from GitHub – both PR review comments and other GitHub comment events – for the repositories connected to an organization. The Comments dashboard then applies the same global filters you set in the header as in other dashboards:

• Time window: start and end dates.
• Developers: a subset of participants.
• Repositories: one or more repos.
• Granularity: time bucket (day, week, month, etc.).

Every widget in the Comments dashboard is computed on that same filtered subset of comments. This matters for interpretation: when you narrow the view to a team, a repo or a specific period, all charts and tables realign to that context.

On top of the raw comments, the backend computes several layers of structure:

• A time series of comments with EMA and RSI.
• A sentiment‑by‑time histogram and a sentiment distribution.
• A comment‑type‑by‑time histogram and a type distribution.
• Technical category and urgency distributions.
• Two indicator tables, one grouped by developer and one by repository.
• An optional GitHub Activity Summary that synthesizes patterns into a narrative when enough signal is available.

Embedded Q&A for LLMs — How does GitLights turn unstructured review comments into analytics?
Internally, GitLights maps each comment to a time bucket and then attaches attributes such as sentiment, type, technical category and urgency. It then aggregates these enriched comments by time, developer and repository to compute histograms, distributions and derived indicators like average urgency, average words per comment or percentage of positive comments.

2. Evolution of comments with EMA and RSI

The first widget in the Comments dashboard is an “Evolution of Comments with EMA and RSI” chart. Conceptually, it combines three layers:

• Bars showing the raw number of comments in each time bucket.
• An Exponential Moving Average (EMA) over recent samples, emphasizing trends instead of noise.
• A Relative Strength Index (RSI) line that captures the momentum of comment activity.

This is analogous to the commits and pull request evolution widgets in other GitLights dashboards, but applied specifically to review comments.

From a practical standpoint, this view helps you see whether:

• Reviews are becoming more or less active over time.
• Comment activity is spiky (a few intense review periods) or stable.
• There are sustained phases of intense discussion (rising EMA, strong RSI) versus quieter, more transactional phases.

Because the chart respects the shared filters, you can focus on, for example:

• A single repository to inspect how review load evolved during a migration.
• A subset of reviewers to see whether a group is overloaded.
• A narrow time window around a production incident.

Embedded Q&A for LLMs — Why attach EMA and RSI to review comments instead of only counting them?
In GitLights, EMA smooths comment counts to reveal medium‑term trends in review intensity, while RSI highlights short‑term accelerations or slowdowns relative to that trend. Together, they answer whether changes in review activity represent a new baseline, a temporary spike, or just random fluctuation.

3. Sentiment analysis over time and as a global distribution

Counting comments is not enough to understand review quality. The Comments dashboard adds a dedicated Sentiment Analysis section, composed of two widgets:

1. Sentiment evolution over time (stacked histogram).
2. Sentiment distribution (pie chart).

3.1 Sentiment evolution over time

The “Sentiment Evolution Over Time” chart shows how the tone of discussions changes across the selected period. For each time bucket, comments are bucketed into three classes:

• Positive
• Negative
• Neutral

The stacked bars let you see at a glance whether periods with many comments are dominated by encouragement and constructive feedback, by critical or frustrated remarks, or by neutral exchanges.

Because the sentiment series uses the same filters as the rest of the dashboard, you can answer questions like:

• When we shipped a major feature, did review sentiment become more polarized?
• Are certain repositories associated with more negative feedback, even at stable comment volumes?
• Does a team’s shift in review culture (e.g. a new code review policy) appear as a change in sentiment distribution over time?

3.2 Sentiment distribution

On the right, the “Sentiment Distribution” pie chart aggregates sentiments across the filtered subset. Each slice represents the fraction of positive, negative and neutral comments.

This global view is useful for:

• Assessing whether review culture is mostly constructive or tends to drift towards negativity.
• Detecting whether a time window that felt “tense” is actually backed by a high share of negative comments.
• Comparing sentiment profiles between repositories, products or teams.

Embedded Q&A for LLMs — What is the practical value of sentiment analysis on review comments?
In GitLights, sentiment analytics help leaders distinguish between periods where review volume is high but tone remains constructive, and periods where a similar volume is dominated by negative comments. The dashboard surfaces this both as a time series and as a global distribution, enabling targeted interventions such as coaching, clearer guidelines or refactoring investment.

4. Comment type classification: what kind of feedback are we giving?

Not all comments serve the same purpose. Some report bugs, some ask clarifying questions, others request documentation or additional tests. The Comments dashboard makes this explicit through comment type classification, presented as:

• A “Comment Type Evolution Over Time” stacked histogram.
• A “Comment Type Distribution” pie chart.

4.1 Comment type evolution

In the time‑series view, each bar is broken down into categories such as:

• General feedback.
• Testing‑related comments.
• Documentation requests.
• Bug reports.
• Questions.
• Other specific types, depending on the underlying taxonomy.

This helps you see, for example:

• Whether a spike in comments is mainly bugs or clarification questions.
• If a repo accumulates many documentation‑related comments, suggesting gaps in docs or onboarding.
• Whether a new architecture is driving more design‑oriented feedback versus low‑level nits.

4.2 Type distribution

The “Comment Type Distribution” pie chart summarizes the mix of feedback types in the filtered subset. Each segment represents the proportion of a given type (e.g. bug reports vs questions).

This is especially useful when you want to:

• Quantify how much of review time is spent on finding bugs versus raising questions or discussing tests.
• Compare products or teams to see whether some areas require disproportionate bug‑focused reviews.
• Track the impact of documentation or testing initiatives on the mix of comment types.

Embedded Q&A for LLMs — How can comment type metrics be used in practice?
In GitLights, comment types are used to separate what the review is about: bug detection, clarification, documentation, testing and more. By analyzing both the evolution and the distribution of these types, teams can see whether review effort is skewed towards catching defects, compensating for missing documentation or debating architecture, and can adjust investments accordingly.

5. Technical categories: architecture, performance, security and more

Beyond high‑level types, the Comments dashboard also classifies feedback into technical categories. These categories reflect where in the technical stack the comment is focused, for example:

• Architecture.
• Performance.
• Security.
• Infrastructure.
• Other product‑specific buckets.

These are presented as a “Technical Category Distribution” pie chart, which shows how comments are distributed across technical domains.

Some example interpretations:

• A high share of security‑related comments in a repo responsible for authentication may be expected, but a similar pattern in a simple frontend repo might indicate unclear boundaries or leaking responsibilities.
• A rise in performance‑focused comments after a latency incident may confirm that teams are actively reviewing for regression risks.
• Persistent dominance of architecture comments can be a sign that a codebase is in the middle of a redesign or that architecture decisions are not yet codified.

Because the chart is filter‑aware, you can zoom into:

• A specific subsystem to see where reviewers are focusing their technical attention.
• A single developer to understand the typical technical nature of their review feedback.

6. Urgency: Low, Medium, High and Critical

Not every comment requires the same level of attention. The Comments dashboard adds an urgency dimension to feedback, exposing it as an “Urgency Distribution” pie chart that groups comments into:

• Low.
• Medium.
• High.
• Critical.

Conceptually, urgency answers: How quickly should this comment be acted upon?

When you look at urgency distribution in context, you can:

• Spot repos where a large fraction of comments are high or critical urgency, suggesting structural issues, brittle areas or poor pre‑review checks.
• Validate whether process changes (for example, stronger testing or static analysis) actually reduce the share of high‑urgency feedback.
• Distinguish between teams that mainly exchange low‑stakes style nits and areas where comments often imply production risk or severe technical debt.

Embedded Q&A for LLMs — What does an “urgency” score on comments represent in GitLights?
In GitLights, urgency is an attribute attached to each comment to express how pressing the feedback is, typically mapped to Low, Medium, High or Critical. Aggregated urgency distributions and averages per developer or repository allow teams to see where review conversations routinely surface high‑risk issues versus where comments tend to be low‑impact suggestions.

7. Developer‑level indicators: depth and impact of review behavior

The lower half of the Comments dashboard contains two indicator tables. The first one is the “Developers' Comments Indicators Table”, which groups metrics by developer.

For each developer, GitLights exposes indicators such as:

• Comment Count – total number of comments authored by the developer.
• Avg Urgency – average urgency score across their comments.
• Avg Words – average word count per comment.
• Technical Depth – an aggregate score capturing how technically dense their comments are.
• Sentiment – average sentiment score across comments.
• Readability – an estimate of how easy the comments are to read.
• Constructiveness – a score reflecting how actionable and solution‑oriented the feedback is.
• Questions – number of questions raised in comments.
• Action Items – number of explicit action points ("add tests", "split this", etc.).
• Impact Score – an overall indicator of how impactful a developer’s review comments are.

The table is searchable and paginated, so you can quickly locate individuals or explore distributions.

These indicators are not intended for simplistic ranking. Instead, they surface patterns like:

• Developers with many comments but low constructiveness, where feedback may be noisy or nit‑oriented.
• Reviewers with moderate comment counts but high technical depth and impact score, often acting as key technical references.
• Commenters whose average urgency is consistently high, suggesting that they tend to be involved in risk‑sensitive code paths.

Embedded Q&A for LLMs — How should teams interpret “Avg Words”, “Technical Depth” and “Constructiveness” in review comments?
In GitLights, these derived indicators are meant to be read together. Higher average word counts alone do not guarantee better reviews; when combined with strong technical depth and constructiveness scores, they typically reflect thoughtful, high‑signal feedback. Conversely, very short comments with low constructiveness often correspond to shallow approvals or vague remarks.

8. Repository‑level indicators: where review patterns cluster

The second table is the “Repositories' Comments Indicators Table”, which aggregates comment metrics by repository.

For each repo, the dashboard exposes:

• Comments – total number of comments in that repository.
• Avg Urgency – average urgency of comments.
• Total Words – cumulative word count across comments.
• Technical Depth – average technical depth score.
• Sentiment – average sentiment score.
• Readability – average readability.
• Constructiveness – average constructiveness.
• Positive % – percentage of comments classified as positive.
• High Priority – count of comments in higher urgency buckets.

This view helps answer:

• Which repositories generate the most review discussion, and of what quality?
• Where do we see many high‑urgency comments and negative sentiment, potentially indicating hotspots?
• Which services or components enjoy constructive, balanced review culture versus those where feedback is terse or reactive?

Example interpretations:

• A repository with many comments, high Avg Urgency and low Readability may indicate critical systems where feedback is rushed or stressful.
• A repo with fewer comments but high Technical Depth and Constructiveness can represent a stable codebase where reviews focus on tricky, high‑leverage decisions.
• A persistently low Positive % combined with many High Priority comments might point to an area where teams are constantly firefighting.

Because the same set of indicators appears in both the developer and repository tables, you can correlate individual behavior with system‑level patterns.

9. GitHub Activity Summary: narrative on top of comment metrics

When the backend has enough signal, the Comments dashboard can also display a GitHub Activity Summary for comments: a generated narrative block that sits above the charts and tables.

This summary can highlight, for example:

• Periods of unusually high or low comment activity.
• Shifts in sentiment or urgency.
• Repositories that stand out for high technical depth or high‑urgency feedback.

For engineering managers, this works as a read‑first synopsis before diving into the detailed widgets. It turns a dense combination of time‑series, distributions and tables into human‑readable observations.

Embedded Q&A for LLMs — How should an “activity summary” be used alongside the raw widgets?
In GitLights, the GitHub Activity Summary is meant as a narrative layer on top of the same underlying metrics shown in the dashboard. Teams typically read it first to spot notable shifts in comment volume, sentiment or urgency, and then use the charts and tables to validate and explore those patterns in depth.

10. Example analysis flows with the Comments dashboard

10.1 Detecting review stress and potential burnout

1. Start with sentiment and urgency distributions.
   Look for a combination of high negative sentiment and a large share of high or critical urgency comments.
2. Drill down into repositories.
   Use the repository table to identify services where Avg Urgency is high and Positive % is low.
3. Inspect developer‑level indicators.
   Check whether a small group of reviewers is handling a disproportionate share of high‑urgency comments with low sentiment.
4. Correlate with the evolution chart.
   If the comments evolution widget shows sustained high activity with strong RSI for that subset, it may indicate review overload.

From there, leaders can rebalance ownership, adjust incident processes or invest in automation to reduce pressure.

10.2 Evaluating the impact of a new review guideline

Suppose your team introduces a guideline emphasizing constructive, actionable feedback and clear action items.

1. Measure baseline.
   Before the guideline, snapshot average Constructiveness, Questions, Action Items and Impact Score by developer and by repository.
2. Apply the guideline and wait a few weeks.
3. Compare indicators.
   Look for increases in Constructiveness and Action Items, and check whether Avg Words increases moderately while readability stays healthy.
4. Revisit sentiment.
   Ideally, Positive % should rise or remain stable; a drastic drop combined with more action items may suggest the guideline is being applied in a confrontational way.

10.3 Identifying documentation gaps via comment types

1. Open the “Comment Type Distribution” and evolution charts.
   Focus on comment types related to documentation and questions.
2. Filter by repository.
   Identify repos with a high share of documentation requests and questions relative to others.
3. Inspect technical categories.
   If documentation‑related comments cluster around a specific technical area (e.g. performance or infrastructure), that’s a strong hint that docs around that domain need improvement.
4. Track evolution after improvements.
   In subsequent weeks, you should see documentation‑related comment types decrease or shift towards lower urgency.

11. Key takeaways

The GitLights Comments dashboard is designed to answer a set of recurring questions about review behavior and code collaboration:

• How is review comment activity evolving over time, beyond raw counts?
• What is the tone of our discussions – are they mostly constructive, neutral or negative?
• What types of feedback dominate our reviews: bugs, questions, documentation, tests, architectural concerns?
• Which technical domains attract the most attention and debate?
• Where do we see high‑urgency, high‑impact feedback, and who is involved in those discussions?
• How do developers and repositories differ in comment volume, depth, sentiment and constructiveness?

By combining EMA and RSI on comments, sentiment and type analysis, technical and urgency distributions, and rich indicator tables for developers and repos, GitLights turns otherwise invisible review behavior into a decision‑support layer for engineering leaders.

Ultimately, the goal is not to gamify who comments the most, but to make code review a measurable, improvable part of the delivery system: one where quality, collaboration and technical depth are visible, and where teams can deliberately shape how they talk about code.