Skip to content

SVD very very slow and GELS gives nans, -inf #11174

Closed
@danielhanchen

Description

@danielhanchen

Hey PyTorch devs! First ever issue from me! :)
Anyways, I was investigating the speed of many libraries including Numpy, Numba JIT, PyTorch, Scipy, and figuring out which libraries perform the best on 3 tasks so I could make ML faster for HyperLearn! https://github.com/danielhanchen/hyperlearn/.

  1. SVD / Pseudoinverses
  2. Eigh, Eigenvalue decomp
  3. Lstsq Solve, Least Squares solving.

Results in seconds are below given N=5,000 and P=6,000

image

Now, as you can see for all functions, PyTorch is slower. SVD is really shocking, taking approx 2 or so minutes, whilst Scipy takes approx 30ish seconds. I'm guessing it's because I used the divide and conquer SVD, and assuming that PyTorch is not divide and conquer (hence maybe why I'm seeing not >60% CPU usage --> I set num_threads == max, I'm only seeing 30%)

Also, Eigh is ok. It's very close to Numpy's Eigh, but still a bit suspicious for XXT, where over 10seconds difference is seen.

Finally, Gels is really bad. I do have to disclose I am using a rank deficient matrix (maybe that's why gels is failing?), but there are nans and -infs. I'm assuming division by 0 is causing the error. If you can't fix this, maybe placing theta_hat[ np.isnan(theta_hat) | np.isinf(theta_hat) ] = 0 can solve 1/2 of the problem. However, Gels is surprisingly fast when compared to say Scipy / Numpy's lstsq. Just its super unstable and MSE of the results are really high.

Anyways great work on the package! An extra longer prelim results is @ unslothai/hyperlearn#7

  • PyTorch or Caffe2: PyTorch
  • How you installed PyTorch (conda, pip, source): conda
  • Build command you used (if compiling from source): --
  • OS: Windows 10
  • PyTorch version: 0.4.1 --> used set_num_threads to max
  • Python version: 3.6.6
  • CUDA/cuDNN version: -- (using CPU)
  • GPU models and configuration: --
  • GCC version (if compiling from source): --
  • CMake version: --
  • Versions of any other relevant libraries: MKL latest (i hope)

cc @jianyuh @nikitaved @pearu @mruberry @heitorschueroff @walterddr @IvanYashchuk @VitalyFedyunin @ngimel

Metadata

Metadata

Assignees

No one assigned

    Labels

    module: linear algebraIssues related to specialized linear algebra operations in PyTorch; includes matrix multiply matmulmodule: performanceIssues related to performance, either of kernel code or framework gluetriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions