Open
Description
Steps to reproduce
- Create a new, empty flutter project with 3.32.0:
flutter create -e timeout_test - Add the integration test dependency:
dart pub add "dev:integration_test:{sdk: flutter}" - Add a basic integration test (see example below)
- Follow the guide from here to setup testing via XCTest: https://github.com/flutter/flutter/tree/main/packages/integration_test#ios-device-testing
- Run the integration tests either via XCode or by building the test bundle and that starting it via the CLI - the result is always the same
- Running via firebase testlab produces the same results
Expected results
The integration tests run normally and succeed.
Actual results
The actual result differs depending on how the test are run:
- Via
flutter test: Succeed as expected - Via XCode oder xcodebuild CLI
- For the simulator, something very weird happens. The simulator starts once, executing the test but never closing the app afterwards. Shortly after the tests have "finished" (the testcases are done, but the app stays open), a second simulator is launched, which starts the app and the tests and actually closes the app again. However, after this there are two simulators, the first still with the app open, the second one done. The test is now stuck and times out after 5 minutes
- For devices, the result is the same as for the first simulator: The app executes all tests but then gets stuck, not closing the app and never finishing.
- Via firebase test labs: Same as 2
The same setup worked flawlessly with flutter 3.29.0
Note: I attached logs below, but the on device logs give the "Warning: integration_test plugin was not detected." That might be relevant.
Code sample
Code sample
main.dart must not be modified. I created the sample with flutter create -e test_timeout. The I added the following integration test as integration_test/app_test.dart:
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import 'package:test_timeout/main.dart' as app;
void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
testWidgets("failing test example", (WidgetTester tester) async {
app.main();
await tester.pumpAndSettle();
expect(find.text('Hello World!'), findsOne);
});
}Next, simply follow the guide from https://github.com/flutter/flutter/tree/main/packages/integration_test#ios-device-testing
I Also attached the zipped test project: test_timeout.zip
Screenshots or Video
Screenshots / Video demonstration
test_timeout_sample.mp4
Logs
XCode Test Logs (Simulator)
-- LLDB integration loaded --
FlutterView implements focusItemsInRect: - caching for linear focus movement is limited as long as this view is on screen.
flutter: The Dart VM service is listening on http://127.0.0.1:53977/Pjhmq4Qgbbw=/
Failed to send CA Event for app launch measurements for ca_event_type: 0 event_name: com.apple.app_launch_measurement.FirstFramePresentationMetric
flutter: 00:00 +0: failing test example
[ERROR:flutter/shell/platform/darwin/ios/framework/Source/FlutterDartVMServicePublisher.mm(129)] Could not register as server for FlutterDartVMServicePublisher, permission denied. Check your 'Local Network' permissions for this app in the Privacy section of the system Settings.
Failed to send CA Event for app launch measurements for ca_event_type: 1 event_name: com.apple.app_launch_measurement.ExtendedLaunchMetrics
flutter: 00:00 +1: (tearDownAll)
flutter: Warning: integration_test plugin was not detected.
flutter:
flutter: If you're running the tests with `flutter drive`, please make sure your tests
flutter: are in the `integration_test/` directory of your package and use
flutter: `flutter test $path_to_test` to run it instead.
flutter:
flutter: If you're running the tests with Android instrumentation or XCTest, this means
flutter: that you are not capturing test results properly! See the following link for
flutter: how to set up the integration_test plugin:
flutter:
flutter: https://docs.flutter.dev/testing/integration-tests
flutter:
flutter: 00:03 +2: All tests passed!Firebase Test Lab Logs (Real Device)
2025-06-06_03_42_13.029851_bhfa_iphone13pro-15.7-en-portrait_syslog.txt
Note: That run is from another example, but has exactly the same issues. Search for iPhone Runner(Flutter)[3836] <Notice>: in the logs to find the relevant parts
Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.32.2, on macOS 15.5 24F74 darwin-arm64, locale
de-DE) [730ms]
• Flutter version 3.32.2 on channel stable at /Users/felix.barz/sdk/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 8defaa71a7 (2 days ago), 2025-06-04 11:02:51 -0700
• Engine revision 1091508939
• Dart version 3.8.1
• DevTools version 2.45.1
[✓] Android toolchain - develop for Android devices (Android SDK version 35.0.1)
[3,1s]
• Android SDK at /Users/felix.barz/sdk/android
• Platform android-35, build-tools 35.0.1
• ANDROID_HOME = /Users/felix.barz/sdk/android
• ANDROID_SDK_ROOT = /Users/felix.barz/sdk/android
• Java binary at: /Applications/Android
Studio.app/Contents/jbr/Contents/Home/bin/java
This is the JDK bundled with the latest Android Studio installation on
this machine.
To manually set the JDK path, use: `flutter config
--jdk-dir="path/to/jdk"`.
• Java version OpenJDK Runtime Environment (build 21.0.6+-13368085-b895.109)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 16.4) [6,0s]
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 16F6
• CocoaPods version 1.16.2
[✓] Chrome - develop for the web [11ms]
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2024.3) [11ms]
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 21.0.6+-13368085-b895.109)
[✓] VS Code (version 1.100.3) [9ms]
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.112.0
[✓] Connected device (2 available) [10,7s]
• macOS (desktop) • macos • darwin-arm64 • macOS 15.5 24F74 darwin-arm64
• Chrome (web) • chrome • web-javascript • Google Chrome 137.0.7151.69
[✓] Network resources [295ms]
• All expected network resources are available.
• No issues found!