X-DeepLearning (XDL for short) is a complete set of deep optimization solutions for high-dimensional sparse data scenarios (such as advertising/recommendation/search, etc.). XDL version 1.2 has been released recently. Performance optimization for large batch/low concurrency scenarios, 50-100% performance improvement in such scenarios. Storage and communication optimization, parameters are automatically allocated globally without manual intervention, and requests are merged to completely eliminate computing/storage/communication hotspots of ps. Complete streaming training features including feature admission, feature elimination, model incremental export, feature counting statistics, etc. Background: XDL1.0 focuses on throughput optimization and adopts the one request per thread processing model, which can significantly improve the limit throughput under ultra-high concurrency.

Features

  • Performance optimization of large batch/single-sample massive feature scenarios
  • Parameter assignment and communication optimization
  • Unified storage and average distribution of parameters: ensure the average distribution of computing, communication, and memory on all servers
  • Automatically analyze and merge undependent communication nodes in the calculation graph to reduce the number of communications and improve communication efficiency
  • Simplify user usage costs, no longer need to provide the estimated value of the embedding parameter size, and no longer need to do regular rebalance
  • In scenarios with massive features, performance and scalability are further improved

Project Samples

Project Activity

See All Activity >

License

Apache License V2.0

Follow X-DeepLearning

X-DeepLearning Web Site

Other Useful Business Software
Get Avast Free Antivirus | Your top-rated shield against malware and online scams Icon
Get Avast Free Antivirus | Your top-rated shield against malware and online scams

Boost your PC's defense against cyberthreats and web-based scams.

Our antivirus software scans for security and performance issues and helps you to fix them instantly. It also protects you in real time by analyzing unknown files before they reach your desktop PC or laptop — all for free.
Free Download
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of X-DeepLearning!

Additional Project Details

Registered

2022-02-02