The Wayback Machine - https://web.archive.org/web/20210724005808/https://github.com/topics/data-engineering
Skip to content
#

data-engineering

Here are 888 public repositories matching this topic...

marvin-robot
marvin-robot commented May 17, 2021

Opened from the Prefect Public Slack Community

elliot: Hey folks is anyone else getting a deprecation warning on marshmallow? Something like:

...
/home/rof/.pyenv/versions/3.9.1/lib/python3.9/site-packages/marshmallow/fields.py:198
/home/rof/.pyenv/versions/3.9.1/lib/python3.9/site-packages/marshmallow/fields.py:198: RemovedInMarshmallow4Warning: Passing fi
lakeFS
nopcoder
nopcoder commented Jul 1, 2021

Currently lakeFS register openapi handlers and handle all specific routes.
In case of a call to /api/v1/test, the unknown path under the API prefix, the mux will serve the request by the UI handler and return a valid HTML (UI) page.

The expected behaviour is to return a non-2xx status code with JSON error - prefered the internal error format, so the developer will handle an error and not fai

A comprehensive list of 180+ YouTube Channels for Data Science, Data Engineering, Machine Learning, Deep learning, Computer Science, programming, software engineering, etc.
  • Updated Jul 3, 2021
anks7190
anks7190 commented Jan 27, 2021

Hi ,

I am using some basic functions from pyjanitor such as - clean_names() , collapse_levels() in one of my code which I want to productionise.
And there are limitations on the size of the production code base.
Currently ,if I just look at the requirements.txt for just "pyjanitor" , its huge .
I don't think I require all the dependencies in my code.
How can I remove the unnecessary ones ?

davidradl
davidradl commented Jul 8, 2021

In the repository handler

  • removeEntity tries to delete then if delete is not supported issues a purge, the purge method issues an audit log
  • There are 2 callers to purgeRelationship only one of which audit logs

This is inconsistent.
I suggest we move the relationship audit log to the purge method, which means that both callers will audit log.

Improve this page

Add a description, image, and links to the data-engineering topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the data-engineering topic, visit your repo's landing page and select "manage topics."

Learn more