- Implemented MonitorUtil for managing monitor secrets and populating them in monitor steps and tests.
- Created StackTraceParser to parse and structure stack traces from various programming languages.
- Developed SyslogParser to handle and parse syslog messages in both RFC 5424 and RFC 3164 formats.
- Added ManualAPI for manually triggering workflows via GET and POST requests.
- Introduced WorkflowAPI for updating workflows with authorization checks.
- Created documentation for JavaScript and Webhook components.
- Established WorkflowFeatureSet to initialize routing and job processing.
- Developed QueueWorkflow service for managing workflow queue operations.
- Implemented RunWorkflow service to execute workflows with error handling and logging.
- Added utility for loading component metadata dynamically.
- Deleted the otel-collector job from GitHub workflows and related deployment configurations.
- Updated Helm charts to remove references to the OpenTelemetry Collector, including its deployment and service definitions.
- Added gRPC server functionality directly in the telemetry service to handle telemetry ingestion.
- Updated environment variables and Docker configurations to reflect the removal of the OpenTelemetry Collector.
- Adjusted telemetry service to support both HTTP and gRPC protocols for telemetry data.
- Removed workflow service from docker-compose files.
- Updated environment configuration to remove workflow port.
- Implemented new ComponentCodeAPI, ManualAPI, and WorkflowAPI for handling workflow operations.
- Created QueueWorkflow service for managing workflow queue operations.
- Developed RunWorkflow service to execute workflows with improved error handling and logging.
- Added component metadata loading utility for dynamic component management.
- Introduced documentation for JavaScript and Webhook components.
- Enhanced overall structure and organization of workflow feature set.
- Created NotFound.ejs for 404 error page with user-friendly messaging and navigation.
- Added ServerError.ejs for 500 error handling with retry and documentation links.
- Introduced Content.ejs partial for structured article content display.
- Developed Head.ejs partial for consistent head elements across pages.
- Implemented Header.ejs partial for navigation and branding.
- Created Nav.ejs partial for sidebar navigation with dynamic links.
- Added OpenSourceCommitment.ejs partial to highlight open-source contributions.
- Implemented Pagination.ejs partial for navigation between documentation sections.
- Updated package.json in StatusPage to correct dependency path for Common.
- Enhanced main package.json with new scripts for building and watching frontend applications.
- Modified tsconfig.json to exclude frontend directories from compilation.
- Simplified Nginx configuration by removing individual upstreams for each service and routing all to a single app upstream.
- Refactored configure.sh to streamline Dockerfile generation.
- Cleaned up docker-compose files by consolidating services and removing unnecessary definitions.
- Introduced new frontend handling logic in Index.ts for rendering different frontend applications.
- Added utility functions for managing status page data and RSS feeds in StatusPage.ts.
- Created dev.sh and frontend-run.sh scripts to facilitate development and build processes for frontend applications.
- Deleted package.json and tsconfig.json for IsolatedVM.
- Removed isolated-vm upstream configuration from Nginx default.conf.template.
- Removed ISOLATED_VM_PORT and telemetry settings from config.example.env.
- Cleaned up docker-compose.base.yml by removing isolated-vm service and its environment variables.
- Updated docker-compose.dev.yml to eliminate isolated-vm service and its associated volumes.
- Removed isolated-vm service from docker-compose.yml.
- Added McpTypes.ts for defining JSON schema and tool information.
- Created ModelType.ts and OneUptimeOperation.ts enums for model types and operations.
- Implemented MCPLogger for structured logging in MCP servers.
- Integrated MCP routes into the main application index.
- Updated package.json to include @modelcontextprotocol/sdk dependency.
- Removed MCP-related configurations from Helm chart and Docker Compose files.
- Added Data Processing Agreement (DPA) page and route to the legal section.
- Updated legal.ejs to include a link to the new DPA page.
- Implemented Probe registration endpoint to handle global probes.
- Created Server Monitor ingestion API with endpoints for secret key verification and response ingestion.
- Developed job processing functions for incoming request and probe ingestion.
- Added middleware for probe authorization and request type definitions.
- Enhanced monitor utility functions to populate secrets in monitor steps.
- Introduced queue stats and failed jobs endpoints for server monitor.
- Deleted RunCommandTool, SearchWorkspaceTool, WriteFileTool, and their associated interfaces and implementations.
- Removed Tool, ToolRegistry, and AgentLogger classes, along with their dependencies.
- Eliminated utility functions for workspace path management and secret sanitization.
- Cleaned up TypeScript configuration and example environment variables related to Copilot.
- Updated Docker Compose files to remove references to Copilot services.
- Add Telemetry service entrypoint
- Telemetry/Index.ts: app bootstrap, routes mounting, infrastructure init and Telemetry SDK init.
- Unified queue + worker
- Telemetry/Jobs/TelemetryIngest/ProcessTelemetry.ts: single worker that dispatches queued jobs to specific processors (logs, traces, metrics, syslog, fluent logs).
- Telemetry/Services/Queue/TelemetryQueueService.ts: central queue API and job payload types.
- Per-type Queue wrappers (LogsQueueService, MetricsQueueService, TracesQueueService, FluentLogsQueueService, SyslogQueueService).
- OpenTelemetry ingestion middleware and proto support
- Telemetry/Middleware/OtelRequestMiddleware.ts: detect OTLP endpoint (logs/traces/metrics), decode protobuf bodies using protobufjs and set product type.
- Telemetry/ProtoFiles/OTel/v1/*.proto: include common.proto, logs.proto, metrics.proto, resource.proto, traces.proto for OTLP v1 messages.
- Ingest services
- Telemetry/Services/OtelLogsIngestService.ts: parse incoming OTLP logs, map attributes, convert timestamps, batch insert logs.
- Telemetry/Services/OtelTracesIngestService.ts: parse OTLP traces, build span rows, extract exceptions, batch insert spans and exceptions, save telemetry exception summary.
- Telemetry/Services/OtelMetricsIngestService.ts: parse OTLP metrics, normalize datapoints, batch insert metrics and index metric name -> service map.
- Telemetry/Services/SyslogIngestService.ts: syslog ingestion endpoints, parser integration, map syslog fields to attributes and logs.
- Telemetry/Services/FluentLogsIngestService.ts: ingest Fluentd style logs, normalize entries and insert into log backend.
- Telemetry/Services/OtelIngestBaseService.ts: helpers to resolve service name from attributes/headers.
- Syslog parser and utilities
- Telemetry/Utils/SyslogParser.ts: robust RFC5424 and RFC3164 parser, structured data extraction and sanitization.
- Telemetry/Tests/Utils/SyslogParser.test.ts: unit tests for parser behavior.
- Telemetry exception utilities
- Telemetry/Utils/Exception.ts: generate exception fingerprint and upsert telemetry exception status (saveOrUpdateTelemetryException).
- Queue & job integration
- New integration with Common/Server/Infrastructure/Queue and QueueWorker, job id generation and telemetry job types.
- Telemetry services add ingestion jobs instead of processing synchronously.
- Config, build and dev tooling
- Add Telemetry/package.json, package-lock.json, tsconfig.json, nodemon.json, jest config.
- New script configs and dependencies (protobufjs, ts-node, jest, nodemon, etc).
- Docker / environment updates
- docker-compose.base.yml, docker-compose.dev.yml, docker-compose.yml: rename service from open-telemetry-ingest -> telemetry and wire TELEMETRY_* envs.
- config.example.env: rename and consolidate environment variables (OPEN_TELEMETRY_* -> TELEMETRY_*, update hostnames and ports).
- Tests/Scripts/status-check.sh: update ready-check target to telemetry/status/ready.
- Other
- Telemetry/Services/Queue/*: export helpers and legacy-compatible job interface shims.
- Memory cleanup and batching safeguards across ingest services.
- Logging and capture spans added to key code paths.
BREAKING CHANGES / MIGRATION NOTES:
- Environment variables and docker service names changed:
- Replace OPEN_TELEMETRY_... vars with TELEMETRY_... (PORT, HOSTNAME, CONCURRENCY, DISABLE_TELEMETRY, etc).
- docker-compose entries moved from "open-telemetry-ingest" to "telemetry" and image name changed to oneuptime/telemetry.
- Update any deployment automation and monitoring checks referencing the old service name or endpoints.
- Consumers: OTLP endpoints and behavior remain supported, but ingestion is now queued and processed asynchronously.
Testing / Running:
- Install deps in Telemetry/ (npm install) after syncing Common workspace.
- Run dev: npx nodemon (nodemon.json) or build & start using provided scripts.
- Run tests with jest (Telemetry test suite includes SyslogParser unit tests).
Files added/modified (high level):
- Added many files under Telemetry/: Index, Jobs, Middleware, ProtoFiles, Services, Utils, Tests, package and config artifacts.
- Modified docker-compose.* and config.example.env and status check script to use new TELEMETRY service/vars.
- Move Fluent/Fluent Bit logs ingestion into open-telemetry-ingest:
- Add OpenTelemetryIngest/API/Fluent.ts (routes for /fluentd and queue endpoints)
- Add Queue service, job worker and processor:
- OpenTelemetryIngest/Services/Queue/FluentLogsQueueService.ts
- OpenTelemetryIngest/Jobs/TelemetryIngest/ProcessFluentLogs.ts
- Register Fluent API and job processing in OpenTelemetryIngest/Index.ts
- Introduce QueueName.FluentLogs and related queue usage
- Remove legacy FluentIngest service and configuration:
- Delete fluent-ingest docker-compose/dev/base entries and docker-compose.yml service
- Remove fluent-ingest related helm values, KEDA scaledobject, ingress host and schema entries
- Remove FLUENTD_HOST env/values and replace FLUENT_INGEST_HOSTNAME -> FLUENT_LOGS_HOSTNAME (pointing to open-telemetry-ingest)
- Update config.example.env keys (FLUENT_LOGS_CONCURRENCY, DISABLE_TELEMETRY_FOR_FLUENT_LOGS)
- Remove FluentIngestRoute and FLUENT_INGEST_URL/hostname usages from UI config/templates
- Remove VSCode launch debug config for Fluent Ingest
- Remove Fluent ingest E2E status check entry in Tests/Scripts/status-check.sh
- Update docs/architecture diagram and Helm templates to reflect "FluentLogs" / Fluent Bit flow
- Misc:
- Remove FLUENTD_HOST environment injection from docker-compose.base.yml
- Cleanup related values.schema.json and values.yaml entries
This consolidates log ingestion under the OpenTelemetry ingest service and removes the separate FluentIngest service and its configuration.
This commit updates the react-syntax-highlighter package to the latest version and switches to the a11yDark style for syntax highlighting in the MarkdownViewer component. This update ensures compatibility with the latest features and improvements of react-syntax-highlighter and enhances the accessibility of the code snippets in the MarkdownViewer.
This commit updates the Llama app to use a local model path instead of a model ID. The model path is set to "/app/Models/Meta-Llama-3-8B-Instruct". This change improves the reliability and performance of the app by directly referencing the model file instead of relying on an external model ID.
This commit modifies the Dockerfile.tpl file to update the EXPOSE directive. The port number is changed from 80 to 8547 to align with the port used by the Llama application. This change ensures that the Llama application is accessible from outside the container on the correct port.
