-
Notifications
You must be signed in to change notification settings - Fork 540
Add lazy EMD solver with O(n) memory requirement #788
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
- Implement emd_c_lazy in C++ network simplex for memory-efficient OT - Add lazy mode to emd2() accepting coordinates (X_a, X_b) instead of cost matrix - Support sqeuclidean, euclidean, and cityblock metrics - Add __restrict__ for SIMD optimization - Remove debug output from network_simplex_simple.h - Add tests for lazy solver and metric variants
test/test_solvers.py
Outdated
| X_sb, X_tb, ab, bb = nx.from_numpy(X_s, X_t, a, b) | ||
|
|
||
| # Test all supported metrics | ||
| for metric in ["sqeuclidean", "euclidean", "cityblock"]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use @pytest.mark.parametrize to check teh metrics instead of a loop inside test
| return G | ||
|
|
||
|
|
||
| def emd2( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is too much of an API change please create a new function emd2_lazy
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #788 +/- ##
==========================================
- Coverage 97.07% 97.03% -0.04%
==========================================
Files 107 107
Lines 22156 22249 +93
==========================================
+ Hits 21507 21590 +83
- Misses 649 659 +10 🚀 New features to boost your workflow:
|
Add lazy EMD solver with on-the-fly distance computation
Types of changes
Motivation and context / Related issue
Addresses memory limitations when computing OT with large point clouds. Instead of pre-computing and storing the full n×m cost matrix, the lazy solver computes distances on-the-fly during the network simplex algorithm. This reduces memory from O(nm) to O(n+m) while maintaining exact EMD solutions.
How has this been tested
test_solvers.py:test_solve_sample_lazyandtest_solve_sample_lazy_emdPR checklist