This passes locally, but seems like it's always timing out on Travis, probably due to having to use the VM builds rather than the docker container builds for python 3.7. This is what sudo: yes is implicitly doing here:
https://github.com/dpkp/kafka-python/pull/1614/files#diff-354f30a63fb0907d4ad57269548329e3R21

I'm hesitant to merge if we're going to constantly hit timeouts because we don't want to be fighting travis trying to make PRs go green... I'll try again tomorrow and see if I continue to hit this.

See also #1549, this seems to be a valid issue but doesn't appear until 3.7.

I feel like we have a bit too many jobs for our test runners. The PR seems good, maybe lets run it on a more recent Kafka version (>=0.11), that may (maybe) help with timeouts.

I feel like we have a bit too many jobs for our test runners.

And we need to add the 2.X Kafka brokers, so it will only get worse... I have thought several times about cutting down, but I am hesitant to do so on the broker versions as many of those are in production and they do have significant differences... maybe we should drop 3.4/3.5 from the travis matrix as most folks on python 3 tend to stay reasonably up to date

a more recent Kafka version (>=0.11), that may (maybe) help with timeouts.

Oh, I hadn't thought of that, good idea

In aiokafka I recently changed the matrics to only run all kafka versions with latest python and all python versions with latest kafka. Not sure if it fits for kafka-python, maybe just run all brokers for 2.7 and latest python + 1 broker per other python versions if needed.

Example: https://travis-ci.com/aio-libs/aiokafka/builds/88787771

run all brokers for 2.7 and latest python + 1 broker per other python versions.

That sounds reasonable to me. I may not get to it soon, so feel free to do it if you get time (and no worries if you don't).

Also, to put this in context... I typically look at the PR and then decide whether / which tests actually matter... ie, if it's straightforward and just needs to be tested against all brokers, then if some random python 3 version times out, I will often immediately merge.

So IMO it's not too bad to have exhaustive test matrix as long as we're not too hung up on retrying builds that timed out just to make them green if we're comfortable making the judgement call that the scope of the PR doesn't create an edge case that matters for it.

Test failure seems like there's a different java setup for this new test configuration and it is preventing Zookeeper (maybe also Kafka) from starting.

Error: A fatal exception has occurred. Program will exit.Unrecognized VM option 'UseParNewGC'
Error: Could not create the Java Virtual Machine.

Test failure seems like there's a different java setup for this new test configuration and it is preventing Zookeeper (maybe also Kafka) from starting.

That's probably because of the Travis workaround: https://github.com/dpkp/kafka-python/pull/1614/files#diff-354f30a63fb0907d4ad57269548329e3R11

I'm glad it at least got that far, IIRC (it's been a little while) originally I kept seeing timeouts before even getting to that point.

It'd probably be better to switch to the G1GC for all the Java builds that support it. But this isn't something I am focused on right now... first I want to get #1193 and then fix some bugs with the new KafkaAdmin... hopefully have some PRs inbound there in the next few days.

Travis xenial is now official but not the default: https://blog.travis-ci.com/2018-11-08-xenial-release

That makes the workaround, less of a hack so why not try putting these lines at the top of the .travis.yml file:

dist: xenial    # required for Python >= 3.7 (travis-ci/travis-ci#9069)
sudo: required  # required for Python >= 3.7 (travis-ci/travis-ci#9069)

That way we could at least see if KAFKA_VERSION=1.0.1 gets any further than KAFKA_VERSION=0.8.2.2. Or if that is just too much then just reverse the sort order of the env:
section and re-enable the workaround so that KAFKA_VERSION=1.0.1 gets tested.