The Wayback Machine - https://web.archive.org/web/20201201004110/https://github.com/microsoft/cpprestsdk/pull/1110
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http_client_asio based on Botan's TLS implementation #1110

Closed
wants to merge 2 commits into from

Conversation

@hrantzsch
Copy link
Contributor

@hrantzsch hrantzsch commented Apr 15, 2019

Hi,

We are working on an extension for http_client_asio that is based on Botan's TLS implementation rather than openSSL. Would you be interested in integrating this in cpprestsdk?

As Botan's TLS stream strives to be a drop-in replacement for boost::asio::ssl::stream, the amount of required changes is not too big. However, the stream initialization using Botan::TLS::Context is specific to the Botan stream and not a drop-in replacement in that sense. Hence, TLS configurations like certificate validation in http_client_config are not easily transferable.
Note that the TLS stream will probably be available starting from Botan 2.11.0 (see this PR)

We (neXenio) are using cpprestsdk already and will switch our TLS implementation to Botan::TLS::Stream soon. So we would really love to have this upstream.

@msftclas
Copy link

@msftclas msftclas commented Apr 15, 2019

CLA assistant check
All CLA requirements met.

@hrantzsch hrantzsch changed the title Botan client http_client_asio based on Botan's TLS implementation Apr 15, 2019
@garethsb-sony
Copy link
Contributor

@garethsb-sony garethsb-sony commented Apr 16, 2019

It would be nice to have some thought applied to the future of the C++ REST SDK _context classes for http_client, http_listener, ws_client and my ws_listener, especially where they currently expose back-end-specific bits and pieces. Users already have to deal with differences when working with different cpprestsdk build configurations (e.g. WinHttp or Boost.ASIO/OpenSSL) with regard to TLS configuration and some other facets. Adding a third back-end is the time to do something about this.

@@ -128,9 +128,10 @@ endif()
if(CPPREST_HTTP_CLIENT_IMPL STREQUAL "asio")
cpprest_find_boost()
cpprest_find_openssl()
target_compile_definitions(cpprest PUBLIC -DCPPREST_FORCE_HTTP_CLIENT_ASIO)
cpprest_find_botan()
target_compile_definitions(cpprest PUBLIC -DCPPREST_BOTAN_SSL -DCPPREST_FORCE_HTTP_CLIENT_ASIO)

This comment has been minimized.

@BillyONeal

BillyONeal May 7, 2019
Member

Unconditionally switching customers to Botan from OpenSSL doesn't seem like a reasonable change. If you want this to be maintained I think you need to (1) add it as an option rather than unconditionally enable it, and (2) add a configuration verifying that it works to azure-pipelines.yaml. Otherwise maintainers who have never used Botan and have no idea how it works will break it on a regular basis.

Release/src/CMakeLists.txt Outdated Show resolved Hide resolved

#if defined(CPPREST_BOTAN_SSL)
#include <botan/asio_stream.h>
#else

This comment has been minimized.

@BillyONeal

BillyONeal May 7, 2019
Member

Please comment else and endif:

#if defined(CPPREST_BOTAN_SSL)
#else // ^^^ CPPREST_BOTAN_SSL // !CPPREST_BOTAN_SSL vvv
#endif // CPPREST_BOTAN_SSL

This comment has been minimized.

@BillyONeal

BillyONeal May 7, 2019
Member

Occurs elsewhere.

I know the existing codebase is somewhat bad about this, but that's led to several nasty bugs before. I'd like us to follow that convention for substantial new functionality being added like this.

return()
endif()

# TODO: this does not make sure Botan is set and does not fail at configure time

This comment has been minimized.

@BillyONeal

BillyONeal May 7, 2019
Member

Either this todo is an actual todo in which case it should be fixed before we merge this, or it's not an actual todo in which case it should not be committted.

@hrantzsch
Copy link
Contributor Author

@hrantzsch hrantzsch commented May 8, 2019

Hi Billy, thanks for the review. Of course all of these points are very valid--I guess I should have pointed out the draft status of this PR better, sorry about that.

I take it you are generally interested in this?

As of now, the client component can optionally be built with Botan instead of OpenSSL, OpenSSL remaining the default.
Initially our goal was to allow users getting rid of OpenSSL entirely. Unfortunately, OpenSSL will still be required for the http listener component, as the Botan SSL stream cannot (yet) be used for this side.

@hrantzsch hrantzsch force-pushed the hrantzsch:botan-client branch from c060727 to 4d6c900 May 8, 2019
@BillyONeal
Copy link
Member

@BillyONeal BillyONeal commented May 13, 2019

I should have pointed out the draft status of this PR better

Nah, it's fine! We don't check up on PRs here very often, so given that I was over here I wanted to give some feedback in case it'd be helpful.

I take it you are generally interested in this?

Not particularly. The entire cpprestsdk project at this time is kinda in maintenance mode. The usual arrangement for PRs though is for people to contribute things that they care about that the original authors might not care about.

That's why I'm specifically looking for an azure-pipelines.yaml change if you want us to accept this because we super don't know what we're doing when it comes to a different SSL backend.

OpenSSL will still be required for the http listener component

The listener component basically exists to be the unit test harness for the client, so that sounds okay.

@garethsb-sony
Copy link
Contributor

@garethsb-sony garethsb-sony commented May 13, 2019

The listener component basically exists to be the unit test harness for the client, so that sounds okay.

Well that's a case in point where many users care about something the original authors don't. 🙂

We're very much using http_listener for real, with and without TLS, in a number of different platform/scenarios. It has been perfectly functional and, with PRs over the last year or so, has had enough attention that it could quite reasonably be moved out of the namespace experimental IMO. I'd like to contribute the similarly-styled ws_listener we built and which a couple of other users have also reported trying out successfully. I've held back due to (a) incomplete tests, and (b) ideally wanting to refactor some of the things that are in the websocket::client namespace up a layer, since they make sense for both client and listener, and finally therefore because (c) I'm not clear what the level of interest from maintainers is/how high a bar needs to be passed, to invest into doing (a) and (b).

Sorry for jumping on the Botan PR!

@BillyONeal
Copy link
Member

@BillyONeal BillyONeal commented May 13, 2019

We're very much using http_listener for real, with and without TLS, in a number of different platform/scenarios.

Please be advised that it isn't suitable for use in production scenarios -- notably, it has not been security audited, fuzz tested, or otherwise hardened for reasonably safe use in environments wherein the whole internet is attacking you.

it could quite reasonably be moved out of the namespace experimental

We would need to think it was safe to use as a production web server to do that, and we do not believe that is true at this time. (In fact much the opposite -- thread sanitizer has found a number of scary issues that likely require a rewrite of most / all of the listener if we wanted to harden it)

@garethsb-sony
Copy link
Contributor

@garethsb-sony garethsb-sony commented May 13, 2019

Please be advised that it isn't suitable for use in production scenarios -- notably, it has not been security audited, fuzz tested, or otherwise hardened for reasonably safe use in environments wherein the whole internet is attacking you.

Thanks, @BillyONeal, that's a sensible note of caution. We have done some regular pen testing, but I should have clarified we are not yet operating in internet environments.

In fact much the opposite -- thread sanitizer has found a number of scary issues that likely require a rewrite of most / all of the listener if we wanted to harden it

Would it be worth throwing these up as an issue - even if it isn't actively being pursued - to allow users to see these results without having to develop these tests, or sharing the test set up, so that users can easily reproduce the results?

So, what would be the interest/benchmark for adding web::websockets::experimental::listener::ws_listener using websocketpp back-end?

@BillyONeal
Copy link
Member

@BillyONeal BillyONeal commented May 13, 2019

Would it be worth throwing these up as an issue - even if it isn't actively being pursued - to allow users to see these results without having to develop these tests, or sharing the test set up, so that users can easily reproduce the results?

All I did was run the unit tests under tsan. The most common error was assuming multiple threads could concurrently touch shared_ptrs. I fixed a few obvious ones as part of onboarding to Azure Pipelines because without that the tests were too flaky to be useful.

@hrantzsch
Copy link
Contributor Author

@hrantzsch hrantzsch commented May 14, 2019

Ok, thanks a lot to both of you for the feedback.

With that, and with what we have learned from playing around with it more on our side, it has become clear that a larger change than I originally imagined would be required for a clean implementation that works on all platforms. It is probably not desirable for the cpprestsdk project to maintain these changes. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.