text copied from JuliaApproxInference/LikelihoodfreeInference.jl#5
i think this is a better place for this issue, i took the liberty of copying your markdown post @jbrea

ApproxInferenceBase.jl: a common API and some basic utilities

My proposition here is to write together a very light-weight ApproxInferenceBase.jl package that serves as a primary dependency of ABC packages. See for example DiffEqBase.jl or ReinforcementLearningBase.jl for how this is done in other eco-systems. I would include in ApproxInferenceBase.jl

Ingredients

• Basic API to coordinate all algorithms
• everything related to prior distributions
• everything related to summary statistics
• everything related to metrics
• testing (and possibly assertion) utilities
• a well-written documentation of the common API

API

My proposition for the API is the following (I am biased of course, and I am very open to discussion!)

Additional to everything related to priors, summarys stats and metrics,
ApproxInferenceBase.jl exports a function fit! with the following signature

fit!(setup, model, data; verbosity = 0, callback = () -> nothing, rng = Random.GLOBAL_RNG)

Every ABC package that relies on ApproxInferenceBase.jl extends this fit! function, e.g.

ApproxInferenceBase.fit!(method::RejectionABC, model, data; kwargs...) = "blabla"

The user provides models as callable objects (functions or functors) with one argument.
Constants are recommended to be handled with closures.
Extraction of summary statistics is done in the model.
For example

model(params) = "do something with params"

my_complex_model(params, constants) = "do something with params and constants"
model(params) = let constants = "blabla" my_complex_model(params, constants) end

my_raw_model(params) = "returns some raw data"
model(params) = extract_summary_statistics(my_raw_model(params))

struct MyFunctorModel
    options
end
(m::MyFunctorModel)(params) = "do something with m and params"

ABC methods/plans/setups are specified in the form

setup = method(metric = my_metric, kwargs...)
setup = method(prior = my_prior, kwargs...) # if method has a prior

One master packages to access all methods

Similar in spirit to DifferentialEquations.jl we could create one package that aggregates all packages and gives unified access. The dependency graph would be something like

     ApproxInferenceBase.jl
                |
     -----------------------
    |           |          |
 ABCPkg1     ABCPkg2      etc.
    |           |          |
    ------------------------
                |
              ABC.jl

This package does nothing but reexport all the setups/methods defined in the
different packages and the fit! function. The name of this package should of course be discussed.

ABCProblems.jl

I think it would be nice to have a package with typical ABC benchmark problems,
like the stochastic lotka-volterra problem, the blowfly problem etc. Maybe we
could collect them in a package ABCProblems.jl.

New methods to be implemented

Here is an incomplete list of methods that I would love to see implemented in
julia. Together with a collection of benchmark problems one would get a nice box
to benchmark new methods we do research on.

• Expectation Propagation for Likelihood-Free Inference, Barthelmé & Chopin 2014
• Parameter estimation in hidden markov models with intractable likelihoods using sequential monte carlo, Yıldırım et al. 2015
• Approximate Bayesian computation with the Wasserstein distance, Bernton et al. 2019
• Statistical Inference for Generative Models with Maximum Mean Discrepancy, Briol et al. 2019
• Approximate Bayesian Computation with Kullback-Leibler Divergence as Data Discrepancy, Jian et al. 2018
• Automatic Posterior Transformation for Likelihood-Free Inference, Greenberg et al. 2019

Conclusions and Questions

Who would be up for such a collaborative effort?
How do you like my proposition for ApproxInferenceBase.jl? What would you change?
Shall we create ApproxInferenceBase.jl, ABCProblems.jl and ABC.jl? Or something similar with different names?