#
data-integration
Here are 216 public repositories matching this topic...
Upserts, Deletes And Incremental Processing on Big Data.
bigdata
stream-processing
data-integration
datalake
apachespark
hudi
apachehudi
incremental-processing
apacheflink
-
Updated
Jul 24, 2021 - Java
Community-curated list of software packages and data resources for single-cell, including RNA-seq, ATAC-seq, etc.
python
bioinformatics
analysis
clustering
gene-expression
data-visualization
dimensionality-reduction
awesome-list
data-integration
atac-seq
single-cell
rna-seq-data
scrna-seq-data
cell-cycle
cell-differentiation
gene-expression-profiles
analysis-pipeline
cell-populations
rna-seq-experiments
cell-clusters
-
Updated
Jul 6, 2021
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
-
Updated
Jul 10, 2021 - Python
xtreding
commented
Jun 14, 2021
Problem
At present, our API documentation and API configuration in Jitsu Server contain some discrepancies. Since Jitsu API grows fast It's unavoidable. We should improve it. Clear documentation is an essential thing in OSS projects.
Solution
We should create Open API specification (yaml file) and generate both Jitsu Server handlers (gin-gonic) and doc
Fast, sensitive and accurate integration of single-cell data with Harmony
-
Updated
Jul 22, 2021 - R
Use SQL to build ELT pipelines on a data lakehouse.
sql
apache-spark
etl
pipelines
data-engineering
data-lake
data-transfer
delta
data-integration
upsert
elt
data-pipeline
datalake
data-ingestion
spark-sql
zeppelin-notebook
apache-iceberg
lakehouse
incremental-updates
-
Updated
Jul 22, 2021 - JavaScript
wisnesky
commented
Sep 11, 2019
Complete algorithm: http://web.cecs.pdx.edu/~mpj/pubs/polyrec.html
NicheNet: predict active ligand-target links between interacting cells
rna-seq
gene-expression
network-inference
data-integration
single-cell-rna-seq
single-cell-omics
intercellular-communication
ligand-receptor
ligand-target
-
Updated
Jun 30, 2021 - R
An example mini data warehouse for python project stats, template for new projects
-
Updated
Jul 21, 2020 - Python
汇总Apache Hudi相关资料
bigdata
apache
stream-processing
data-integration
datalake
hudi
apachehudi
incremental-processing
hudi-resources
-
Updated
Jul 11, 2021
Hetionet: an integrative network of disease
-
Updated
Dec 10, 2020 - HTML
scikit-fusion: Data fusion via collective latent factor models
-
Updated
May 27, 2021 - Python
WInte.r is a Java framework for end-to-end data integration. The WInte.r framework implements well-known methods for data pre-processing, schema matching, identity resolution, data fusion, and result evaluation.
-
Updated
Apr 28, 2021 - Java
Query to reference mapping for single-cell genomics
-
Updated
Jul 23, 2021 - Jupyter Notebook
The Common Core Ontology Repository holds the current released version of the Common Core Ontology suite.
semantics
interoperability
data-integration
ontologies
owl-ontology
bfo
cco
applied-ontology
semantic-consistency
ontology-suite
-
Updated
Apr 23, 2021
Guutch
opened
Jul 22, 2021
-
Updated
Feb 10, 2021 - Python
an data-centric integration platform
-
Updated
Jul 2, 2021 - Java
An Efficient RML-Compliant Engine for Knowledge Graph Construction
-
Updated
Jul 6, 2021 - Python
A .NET class library that allows you to import data from different sources into a unified destination
mysql
html
json
csv
sql-server
csharp
xml
sqlite
excel
tabular-data
data-import
oracle
databases
powerpoint
vcard
data-integration
schema-matching
sqlce
schema-mapping
msaccess
-
Updated
Dec 8, 2020 - C#
Utilities for creating ETL pipelines with mara
-
Updated
Jul 7, 2020 - PLpgSQL
Toolbox for including enzyme constraints on a genome-scale model.
-
Updated
Jul 2, 2021 - MATLAB
First Party data integration solution built for marketing teams to enable audience and conversion onboarding into Google Marketing products (Google Ads, Campaign Manager, Google Analytics).
python
bigquery
google
conversions
dataflow
data-integration
googleanalytics
googleads
audiences
customermatch
audience-targeting
-
Updated
Jul 23, 2021 - Python
Installer for Thymeflow, a personal knowledge management system.
-
Updated
Apr 17, 2018
Research data management in biomedical and machine learning applications
python
workflow
machine-learning
automation
datastructures
neuroscience
pandas
datascience
data-structures
data-integration
neuroimaging
neuroinformatics
biomedical
userfriendly
medical-data
machine-learning-workflows
-
Updated
Apr 2, 2021 - Python
Repo for Data Warehouse Concepts, Design, and Data Integration by University of Colorado System (coursera)(Notes,Assignments, quiz and research papers)
-
Updated
Jun 2, 2018
simulation
systems-biology
computational-biology
data-integration
curation
biological-expression-language
biocuration
knowledge-graph-embeddings
networks-biology
knowledge-graphs
-
Updated
Dec 16, 2019 - TeX
Development of the Gellish Communicator reference application and tools for universal data exchange and data integration supporting Formal English and other Gellish formalized natural languages.
nlp
taxonomies
natural-language-processing
databases
ontology
interoperability
classification
data-integration
family
data-management
reference-implementation
query-language
data-modeling
formal-languages
data-exchange
knowledge-representation
universal-interface
knowledge-modeling
interoperability-of-systems
gellish
formalized-natural-language
-
Updated
Nov 12, 2018 - Python
Improve this page
Add a description, image, and links to the data-integration topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the data-integration topic, visit your repo's landing page and select "manage topics."
Tell us about the problem you're trying to solve
As follow up to airbytehq/airbyte#4721, do proper yaml parsing/extract to be used in https://github.com/airbytehq/airbyte/blob/7decb12818ad2cc0c7b3039c3d5e620236250edd/airbyte-workers/src/main/resources/dbt_transformation_entrypoint.sh#L7
Describe the solution you’d like
we should write (a python?) script that uses