The Wayback Machine - https://web.archive.org/web/20220126161033/https://github.com/microsoft/TypeScript/issues/17861
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crazy checker.ts refactor experiment (25kloc) #17861

Open
Busyrev opened this issue Aug 17, 2017 · 12 comments
Open

Crazy checker.ts refactor experiment (25kloc) #17861

Busyrev opened this issue Aug 17, 2017 · 12 comments

Comments

@Busyrev
Copy link
Contributor

@Busyrev Busyrev commented Aug 17, 2017

I have made tool-assisted refactoring experiment on typeChecker. The goal is to convert checker.ts from incapsulated js scope to a class. For better extensibility and publicity (#17680).

All tests are passing, excluding linting (because of auto converting). We can fix linting issues later.

How I refactored 25k lines of code:

  1. All types moved to ts namespace out of createTypeChecker;
  2. Class TypeCheckerImpl was created.
  3. All variables from createTypeChecker moved to public properties of TypeCheckerImpl;
  4. Property initializers were moved to constructor. (because of constructor parameters dependency)
    declarations now looks ugly, because
    public propName = (false as true) && this.someMethod();
    is only way to auto infer type of function return value; We can fix it later.
  5. All nested functions from createTypeChecker converted to public methods of TypeCheckerImpl;
  6. For all converted functions auto inserted "this" keyword.
  7. For methods that contain nested functions I have to create ugly conv_self to enclosure "this".
  8. Added ugly code to bind all methods. We can fix it later.
  9. Now TypeCheckerImpl instance is not exposed to public, but I want to do it later.

Here is commit in my fork:
Busyrev@697d90e
Unfortunetly GitHub can`t show diff, and that is one more point for necessity of refactoring.

Raw checker.ts before

https://raw.githubusercontent.com/Microsoft/TypeScript/master/src/compiler/checker.ts

after

https://raw.githubusercontent.com/Busyrev/TypeScript/checker-class-public-experimental/src/compiler/checker.ts

This is only experiment.

I have spend around 16 hours for these transforms. I think that this experiment is successful.
For better typescript future I think checker.ts should be separated into multiple files/classes, and this is the first step. This code is not merge ready. Of course we should think about methods to be provided to public api, and may be unsafe access for people who do really complicated transformations.

What do you think about it?

I can dedicate more time, improve documents and publish tools I've made, if more people support this experiment.

@RyanCavanaugh
Copy link
Member

@RyanCavanaugh RyanCavanaugh commented Aug 17, 2017

😱 wow!

Did you see any change in performance?

@Busyrev
Copy link
Contributor Author

@Busyrev Busyrev commented Aug 17, 2017

@RyanCavanaugh I think no. I didn't write performance tests, but all regular typescript tests run in same time as usual build.

@Busyrev
Copy link
Contributor Author

@Busyrev Busyrev commented Aug 18, 2017

got an idea how to prevent splitting property declarations and initialisation, using

constructor(public host: TypeCheckerHost, public produceDiagnostics: boolean )

It fixes problem of properties dependency. I`ll try it

@kitsonk
Copy link
Contributor

@kitsonk kitsonk commented Aug 18, 2017

@Busyrev as built compiler supports tsc --diagnostics and give you some level of indication of performance, if you build something substantial with the two versions.

@andy-ms
Copy link
Contributor

@andy-ms andy-ms commented Aug 22, 2017

I ran our internal performance tests and it looks like there is a large increase in check time. (There were also large increases in emit time, which may just be my hard drive acting up.) There was no change to parse or bind time or memory usage.

Monaco - node (v8.2.1, x64)

Project Baseline Current Delta Best Worst
Memory used 357,006k (± 0.00%) 356,252k (± 0.00%) -754k (- 0.21%) 356,247k 356,258k
Parse Time 2.10s (± 1.10%) 2.10s (± 0.62%) +0.01s (+ 0.43%) 2.09s 2.12s
Bind Time 0.81s (± 1.15%) 0.81s (± 1.61%) +0.01s (+ 0.74%) 0.80s 0.83s
Check Time 4.19s (± 0.60%) 4.89s (± 0.53%) +0.70s (+ 16.62%) 4.86s 4.92s
Emit Time 2.77s (± 0.73%) 6.92s (± 1.01%) +4.15s (+149.96%) 6.88s 7.03s
Total Time 9.87s (± 0.45%) 14.74s (± 0.48%) +4.87s (+ 49.37%) 14.69s 14.84s

Monaco - tsc (x86)

Project Baseline Current Delta Best Worst
Parse Time 1.51s (± 3.64%) 1.48s (± 0.88%) -0.03s (- 1.85%) 1.47s 1.50s
Bind Time 0.58s (± 1.82%) 0.58s (± 2.95%) -0.01s (- 1.03%) 0.56s 0.60s
Check Time 3.98s (± 0.98%) 4.93s (± 0.58%) +0.96s (+ 24.13%) 4.89s 4.95s
Emit Time 7.54s (± 6.52%) 7.24s (± 0.41%) -0.30s (- 3.98%) 7.23s 7.29s
Total Time 13.61s (± 3.93%) 14.24s (± 0.43%) +0.63s (+ 4.64%) 14.16s 14.31s

TFS - node (v8.2.1, x64)

Project Baseline Current Delta Best Worst
Memory used 309,696k (± 0.01%) 308,820k (± 0.01%) -876k (- 0.28%) 308,805k 308,841k
Parse Time 1.53s (± 1.92%) 1.54s (± 4.00%) +0.01s (+ 0.85%) 1.45s 1.58s
Bind Time 0.68s (± 4.95%) 0.69s (± 10.07%) +0.01s (+ 1.02%) 0.66s 0.80s
Check Time 3.60s (± 0.51%) 4.24s (± 0.79%) +0.64s (+ 17.84%) 4.20s 4.28s
Emit Time 2.45s (± 0.33%) 4.09s (± 1.23%) +1.65s (+ 67.31%) 4.06s 4.17s
Total Time 8.26s (± 0.33%) 10.57s (± 0.66%) +2.31s (+ 27.92%) 10.51s 10.66s

TFS - tsc (x86)

Project Baseline Current Delta Best Worst
Parse Time 1.09s (± 5.00%) 1.07s (± 0.90%) -0.02s (- 1.84%) 1.06s 1.08s
Bind Time 0.58s (± 5.24%) 0.57s (± 3.35%) -0.00s (- 0.86%) 0.55s 0.59s
Check Time 3.26s (± 1.15%) 3.95s (± 1.10%) +0.70s (+ 21.44%) 3.90s 3.99s
Emit Time 4.44s (± 1.49%) 4.36s (± 1.21%) -0.08s (- 1.85%) 4.29s 4.41s
Total Time 9.37s (± 1.10%) 9.96s (± 0.86%) +0.59s (+ 6.32%) 9.89s 10.05s

I'd really like to see the checker cut down to size -- it's so big that vscode doesn't support collapsing code, and it takes several seconds to update errors. I think some of the candidates for removal from checker.ts were isTypeAssignableTo and typeToString. @sandersn might have some comments.

@Busyrev
Copy link
Contributor Author

@Busyrev Busyrev commented Aug 22, 2017

@andy-ms thanks for tests! I believe we can beat this 25% slowdown.
Now all methods are bound. I'm working on auto binder now, only for methods used not for calling. It can increase speed, may be. And I`m thinking about removing method createTypeChecker from implementation, and implement Typechecker interface directly.

@andy-ms
Copy link
Contributor

@andy-ms andy-ms commented Aug 31, 2017

Hi @Busyrev , sorry for the delay. It looks like your branch Busyrev/TypeScript/checker-class-public-experimental is still on the Aug 17 commit "First ugly try to refactor checker.ts". I can run performance tests again if you push to that branch.

@chicoxyzzy
Copy link
Contributor

@chicoxyzzy chicoxyzzy commented Jan 5, 2018

That'll be much easier to refactor incrementally if checker.ts would be splitted in modules but it seems there is chicken egg problem here :(

@kevinbarabash
Copy link

@kevinbarabash kevinbarabash commented Dec 9, 2020

An incremental that could be used is to move some of the variables that createTypeChecker defines near the top into a "context" object and then pass that context object to the functions that use those variables. You could start with those variables that are least used. Once a function no longer makes uses of any variables from the closure it can be extracted.

It might even make sense to have a couple of objects. I could very easily seen all of the type intrinsics defined grouped together on their own object. I see that createTypeChecker is called multiple times in program.ts. Is there any worry about reusing types created by createType across multiple calls to createTypeChecker?

@Timmmm
Copy link

@Timmmm Timmmm commented Jan 26, 2021

Can someone explain the performance issues? It seems crazy to me that you can't split up a 41k line file because of that. It makes editing it very difficult - not only the performance issues, but you can't even jump to errors or changes using the scrollbar.

@Busyrev
Copy link
Contributor Author

@Busyrev Busyrev commented Jan 26, 2021

@Timmmm I think that there is two major performanse issues:

  1. all methods are binded. This may be fixed.
  2. using members instead of local variables is slightly slower. And I know no way to speed it up.

@RyanCavanaugh
Copy link
Member

@RyanCavanaugh RyanCavanaugh commented Jan 26, 2021

This gets asked a ton of times, so here's the short version

Imagine you have this code structure

function fn(x) {
  // There are ~450 of these variables
  let y1 = someFunc1(x);
  let y2 = someFunc2(x);

  // There are ~1200 of these functions
  function a() {
    console.log(y1 - x);
  }
  function b(z) {
    console.log(y2 + x * z);
  }
}

It is a fact in modern JS engines that bare-name lookups like y1 and y2 are much faster than property accesses like obj.y1 and obj.y2. The challenge, then: how do you "split up" fn in a way that preserves that performance improvement?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants