mirror of
https://github.com/OneUptime/oneuptime.git
synced 2026-04-06 00:32:12 +02:00
feat(Logs): add missing OpenTelemetry log fields to enhance compliance and debugging
This commit is contained in:
@@ -19,6 +19,7 @@ The following features have been implemented and removed from this plan:
|
||||
- **Phase 2.1** - Saved Views (LogSavedView model, SavedViewsDropdown, CRUD API)
|
||||
- **Phase 2.2** - Log Analytics View (LogsAnalyticsView with timeseries, toplist, table charts; analytics endpoint)
|
||||
- **Phase 2.3** - Column Customization (ColumnSelector with dynamic columns from log attributes)
|
||||
- **Phase 5.8** - Store Missing OpenTelemetry Log Fields (observedTimeUnixNano, droppedAttributesCount, flags columns + ingestion + migration)
|
||||
|
||||
## Gap Analysis Summary
|
||||
|
||||
@@ -197,42 +198,17 @@ These optimizations address fundamental storage and indexing gaps in the telemet
|
||||
- `Common/Server/Utils/AnalyticsDatabase/StatementGenerator.ts` (emit PROJECTION clause)
|
||||
- `Worker/DataMigrations/` (new migration to materialize)
|
||||
|
||||
### 5.8 Store Missing OpenTelemetry Log Fields (Low)
|
||||
|
||||
**Current**: Several standard OTEL log record fields are dropped during ingestion:
|
||||
- `observedTimeUnixNano` — when the log was collected by the pipeline (useful for measuring ingestion lag)
|
||||
- `droppedAttributesCount` — signals data loss during collection
|
||||
- `flags` — log record flags (e.g., W3C trace flags)
|
||||
|
||||
**Target**: Preserve these fields for full OTEL compliance and operational debugging.
|
||||
|
||||
**Implementation**:
|
||||
|
||||
- Add optional columns to the Log model:
|
||||
- `observedTimeUnixNano` (LongNumber)
|
||||
- `droppedAttributesCount` (Number, default 0)
|
||||
- `flags` (Number, default 0)
|
||||
- Update `OtelLogsIngestService.processLogsAsync()` to extract and store these fields
|
||||
- Migration to add columns to existing table
|
||||
|
||||
**Files to modify**:
|
||||
- `Common/Models/AnalyticsModels/Log.ts` (add columns)
|
||||
- `Telemetry/Services/OtelLogsIngestService.ts` (extract additional fields)
|
||||
- `Worker/DataMigrations/` (new migration)
|
||||
|
||||
### 5.x Remaining Performance Impact Summary
|
||||
|
||||
| Optimization | Query Pattern Improved | Expected Speedup | Effort |
|
||||
|-------------|----------------------|-------------------|--------|
|
||||
| 5.3 DateTime64 time column | Sub-second log ordering | Correctness fix | Medium |
|
||||
| 5.7 Histogram projections | Histogram and severity aggregation | 5-10x | Medium |
|
||||
| 5.8 Missing OTEL fields | OTEL compliance | N/A (completeness) | Small |
|
||||
|
||||
### 5.x Recommended Remaining Order
|
||||
|
||||
1. **5.3** — DateTime64 upgrade (correctness)
|
||||
2. **5.7** — Projections (performance polish)
|
||||
3. **5.8** — Missing OTEL fields (completeness)
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user