Kubernetes History Inspector Files

We've migrated the majority of the features from frontend heavy architecture to backend side. This enables us to gather logs even over 10M logs. In addition to this big architecture change, we've implemented full text search index in backend side. This allows us to search resources containing logs with specific keywords really faster. It would return the full text search query for 10M logs in few secounds!

Note I'll skip v0.58.0 and v0.58.1 for OSS release for some reasons. This v0.58.2 is our first release of v0.58.x.

What's Changed

Bug Fixes 🐛

• perf(web): pool and reuse layout components across layout transitions by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/904
• fix(web): update form validation on in-flight requests and migrate to signals by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/907
• fix(progressutil): resolve race condition in ProgressUpdator by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/910
• fix(inspection): fix missing filter resource name fields and query estimation metadata by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/911
• fix(timeline): resolve shader asset paths relative to baseUrl and cache WebGL assets by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/929
• perf(core): optimize orderedMapNode reader access and release raw logs after ingestion by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/931
• fix(csm): return structured.Node from TrackAndGetManifest to fix ResourceBody intern error by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/943
• fix(inspection/googlecloudcommon): avoid race condition dropping partition boundary logs in log fetcher by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/947
• fix(timeline): unify dashed border stripe coordinates to screen pixels by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/949
• Handle truncated Kubernetes audit logs by @Haihan-Jiang in https://github.com/GoogleCloudPlatform/khi/pull/697

Development Environment 💻

• feat(taskrecord): introduce DAG task record and replay test framework by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/924
• docs(skills): add backward-compatibility-policy skill and update coding rules by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/939

Documentation 📝

• docs: add release notes for v0.58.0 by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/927

Other Changes 🛠

• perf: introduce DAG-based Initializer system, eliminate ServerFactory, and remove ViewerMode by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/890
• feat(api,web): introduce Connect-RPC and chunked .khi file import workflow by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/891
• feat: introduce Workbench service with backend CEL filtering and on-demand struct loading by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/892
• feat(khifile): partition client and server intern pools into dedicated chunks by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/893
• feat: display the summary of envoy access logs by @K53 in https://github.com/GoogleCloudPlatform/khi/pull/887
• perf(workbench,filter): optimize filter result payload with SparseBitset and IdBitset by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/894
• feat(inspection): structured log query estimation and inspection log volume UI redesign by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/895
• feat(popup): migrate popup service to Connect-RPC with server-streaming WatchPopup and polymorphic UI registry by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/897
• feat(googlecloud): add adaptive rate limiter for Cloud Logging API queries by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/896
• feat(upload): migrate file parameter upload to Connect-RPC FileParameterUploadService with chunked uploader and async verifier by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/898
• perf(workbench): accelerate regex log filtering with Roaring Bitmap TrigramIndex and optimized SearchIndex by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/899
• docs(agents): add Jujutsu (jj) workflow skill by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/901
• fix(estimator): inject call options in StructuredLogEstimator and document Google API guidelines by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/900
• perf(server): optimize server-side .khi file loading and timeline filtering by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/903
• feat(server_status): separate ServerStatusService into standalone Connect-RPC with CPU usage monitoring by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/902
• feat(workbench): persistent background Trigram indexing and search optimization by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/905
• feat(api): configure Connect-RPC serialization format based on environment by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/906
• feat(inspection): migrate Inspection REST APIs to Connect-RPC and enable HTTP/2 support by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/908
• feat(inspection): add pending status for asynchronous validation and estimation by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/909
• feat(api): add polling fallback APIs for streaming RPCs and transparent interceptor by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/912
• refactor(web): clean up unused frontend store fields and worker remnants by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/913
• feat(web): support dynamic upload/download config overrides and binary transport by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/915
• feat(workbench): introduce timeline severity bitmask cache and hasSeverity CEL function by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/914
• feat(web): tune legacy polling intervals and switch to sequential short polling for async jobs by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/916
• feat(web): detect workbench session expiration and support reopening by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/917
• feat(workbench): generate architecture graph on backend via GetArchitectureGraph RPC by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/918
• feat(workbench): add batch ReadStructYAMLs RPC and client-side prefetching by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/919
• feat(graph): add toolbar menu, image export, and node pod phase resolution by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/921
• feat(parser,store): stream v6 chunks directly into dynamic stores and introduce StructStore by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/920
• perf(workbench): optimize memory footprint and ingestion performance for .khi datasets by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/922
• feat(web): add Connect-RPC retry interceptor and resilient polling/upload/download by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/923
• perf(web): migrate WebGL timeline rendering to IdBitset and optimize selection resolution by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/925
• refactor(structured,parsers): optimize log parsing with FieldPath interning and Extractor pattern by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/928
• feat(logestimator): introduce EstimatedCountPreset for rough query log estimation by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/926
• refactor(web): unify log naming conventions and optimize log list data structure by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/930
• feat(controlplane): support HPA controller logs parsing and timeline mapping by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/932
• perf(core): optimize structured Node fast lookup and flattenNode allocations by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/933
• perf(khifile): optimize InternPool memory retention, ID resolutions, and ToInternedStruct allocations by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/935
• feat(googlecloudcommon): parallelize Cloud Logging conversion with k-way merge and display fetch ETA by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/936
• perf(workbench): pack CEL TrigramKey into uint64 and serialize StructYAMLs on-demand by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/938
• refactor(model): extract id generator into pkg/model/id and use uint32 for log ID by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/937
• perf(core,inspection): parallelize log filtering and optimize filter extractors by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/940
• refactor(model): refactor TimelineBuilder revisions and events to lightweight types by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/941
• perf(patternfinder): optimize Trie memory efficiency with Radix Tree by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/944
• docs(skill): add workspace isolation and prefer-jj guidelines to jj-workflow by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/946
• ci: cache dependencies and share frontend build artifacts across workflows by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/945
• perf(khifile): optimize in-memory model via flat structure-of-arrays representation by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/950
• feat(timeline): extract multi-term log search as shared chip-search-bar dumb component by @kyasbal in https://github.com/GoogleCloudPlatform/khi/pull/951

Full Changelog: https://github.com/GoogleCloudPlatform/khi/compare/v0.58.0...v0.58.2