Description
Hey PyTorch devs! First ever issue from me! :)
Anyways, I was investigating the speed of many libraries including Numpy, Numba JIT, PyTorch, Scipy, and figuring out which libraries perform the best on 3 tasks so I could make ML faster for HyperLearn! https://github.com/danielhanchen/hyperlearn/.
- SVD / Pseudoinverses
- Eigh, Eigenvalue decomp
- Lstsq Solve, Least Squares solving.
Results in seconds are below given N=5,000 and P=6,000
Now, as you can see for all functions, PyTorch is slower. SVD is really shocking, taking approx 2 or so minutes, whilst Scipy takes approx 30ish seconds. I'm guessing it's because I used the divide and conquer SVD, and assuming that PyTorch is not divide and conquer (hence maybe why I'm seeing not >60% CPU usage --> I set num_threads == max, I'm only seeing 30%)
Also, Eigh is ok. It's very close to Numpy's Eigh, but still a bit suspicious for XXT, where over 10seconds difference is seen.
Finally, Gels is really bad. I do have to disclose I am using a rank deficient matrix (maybe that's why gels is failing?), but there are nans and -infs. I'm assuming division by 0 is causing the error. If you can't fix this, maybe placing theta_hat[ np.isnan(theta_hat) | np.isinf(theta_hat) ] = 0 can solve 1/2 of the problem. However, Gels is surprisingly fast when compared to say Scipy / Numpy's lstsq. Just its super unstable and MSE of the results are really high.
Anyways great work on the package! An extra longer prelim results is @ unslothai/hyperlearn#7
- PyTorch or Caffe2: PyTorch
- How you installed PyTorch (conda, pip, source): conda
- Build command you used (if compiling from source): --
- OS: Windows 10
- PyTorch version: 0.4.1 --> used set_num_threads to max
- Python version: 3.6.6
- CUDA/cuDNN version: -- (using CPU)
- GPU models and configuration: --
- GCC version (if compiling from source): --
- CMake version: --
- Versions of any other relevant libraries: MKL latest (i hope)
cc @jianyuh @nikitaved @pearu @mruberry @heitorschueroff @walterddr @IvanYashchuk @VitalyFedyunin @ngimel