Originally created by: erbesharat
Currently, the SplitRunner worker logs are minimal and not very informative, making it difficult to diagnose issues when tasks fail or behave unexpectedly. We propose introducing enhanced, structured logging capabilities within the SplitRunner worker to provide richer context around execution steps, timing, and error conditions. This will involve adding configurable log levels (e.g., debug, info, warn, error) and potentially leveraging a structured logging framework (such as pino or winston) for consistent, JSON-formatted logs. The improved logs should capture key data points like the worker's lifecycle events, the splitting strategies used, the status of tasks in the queue, performance metrics, and detailed error stacks.
By improving the logging, developers and operators can more easily pinpoint the root causes of failures, optimize task processing, and ensure that the SplitRunner worker behaves reliably under load. For example, if a particular task type repeatedly fails at the same processing step, the enhanced logs will help identify patterns, allowing for quicker debugging and faster fixes. This change can benefit all users who rely on SplitRunner for distributed or parallelized processing, particularly in production environments where visibility into complex worker behaviors is essential.
winston) into the SplitRunner module.napi configuration files, allowing users to set the verbosity of logs (e.g., debug vs. info) without code changes.
