Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
add ability to define the size of the database in the constructor #298
Conversation
missvalentinep
changed the title
|
@lovasoa could you pay attention to this change please? This is not just optimization. Thank you! |
|
The structure of this repo makes it very difficult to review changes. I am currently not merging pull requests by lack of time. One thing to do that would make it much easier to make changes is to handle compilation to JS by a continuous integration tool, and maybe use a more popular language such as JS of TypeScript. |
|
What if I removed the compiled files from pull request and you compiled them yourself or left the compilation to prepublish command? |
|
@lovasoa OK, I totally agree. Internet is dangerous place. To be sure there are no malware in minified JS: May I ask you to take diff from |
The fact is we also are tremble because it's hard to understand that |
|
What would both reduce the maintenance burden (which is the biggest problem here) and improve security is having a continuous integration service bundling the files. |
|
@kripken : maybe you can activate GitHub actions for this repo? https://github.com/features/actions |
|
@lovasoa all I see is a way to request being added to the beta - is there something else? I did that now. |
|
No, nothing else. Once you are added to the beta, we can add a yaml file in |
|
I got an email that I am in the beta, so maybe it can work now... |
|
Thank you Alon. I created a github workflow that compiles the code, tests it, and uploads the resulting artifacts: https://github.com/kripken/sql.js/actions |
When we have to work with big data, the ability to predefine the size of a new database can significantly speed up script execution time, because multiple calls to MEMF.expandFileStorage is way less efficient than one "bigger" call to it.
JavaScript profile of merging 7 databases (each of size ~15 MB) into one, without expanding the final db file first
The same operation but with expanding the final database to the size of (sum of 7 databases' sizes)
