Starter for IntelliJ IDEA based IDE's
Overview
The Starter will help you create and run tests for IntelliJ-based IDEs.
It can run an IDE from an existing installer or download one if needed, configure IDE, launch the IDE as an external process, send commands
to the IDE,
collect the logs, metrics and other IDEs output after test execution.
Additional features include:
- Executing commands inside IDE (see the list of available commands below)
- Implementing custom commands for use in tests
- Executing custom code (without relying on external libraries not available in the IDE)
- CI integration (optional)
- Test artifact collection
- Freeze/exception collection
- Reporting artifacts to CI (optional)
- Running tests with a profiler (not yet included)
More details how it works can
be found here
Supported products
- IDEA
- GoLand
- WebStorm
- PhpStorm
- DataGrip
- PyCharm
- RubyMine
- Android Studio
How to use
For the required build.gradle
and various test scenarios, refer to
the Examples directory.
Available commands
The available commands can be found in
the performance-testing-commands package
which includes performance-intensive operations and IDE configuration commands.
Examples of included commands:
- waitForSmartMode()
- flushIndexes()
- setupProjectSdk(sdkName: String, sdkType: String, sdkPath: String)]
- openFile(relativePath: String)
- openProject(projectPath: Path)
- reopenProject()
- goto(line: String, column: String)
- findUsages()
- inspectCode()
- checkOnRedCode()
- exitApp(forceExit: Boolean = true)
- memoryDump()
- dumpProjectFiles()
- compareProjectFiles(firstDir: String, secondDir: String)
- cleanCaches()
- doComplete(times: Int)
- openProjectView()
- pressKey(key: String)
- delayType(command: String)
- doLocalInspection()
- altEnter(intention: String)
- callAltEnter(times: Int, intention: String = "")
- createAllServicesAndExtensions()
- runConfiguration(command: String)
- openFileWithTerminate(relativePath: String, terminateIdeInSeconds: Long)
- searchEverywhere(text: String)
- storeIndices()
- compareIndices()
- recoveryAction(action: RecoveryActionType)
- importMavenProject()
- importGradleProject()
- and many more
Performance testing/Metrics collection
Metrics are values collected during test execution, including operation durations, command results (e.g., the number of elements in
completion lists), and more.
You can find more information about different approach to collect metrics (with or without Starter)
in Starter Readme