mirrord

Fixed

• Add missing dependency breaking the VS Code release.

3.0.0-alpha

Added

• New feature: UDP outgoing, mainly for Go DNS but should work for most use cases also!
• E2E: add tests for python's fastapi with uvicorn
• Socket ops - connect: ignore localhost and ports 50000 - 60000 (reserved for debugger)
• Add "*.plist" to IGNORE_REGEX, refer [#350].

Changed

• Change all functionality (incoming traffic mirroring, remote DNS outgoing traffic, environment variables, file reads) to be enabled by default. Note that flags now disable functionality

Fixed

• mirrord-layer: User-friendly error for invalid kubernetes api certificate
• mirrord-cli: Add random prefix to the generated shared lib to prevent Bus Error/EXC_BAD_ACCESS
• Support for Go 1.19>= syscall hooking
• Fix Python debugger crash in VS Code Extension. Closes [#350].

Added

• Release arm64 agent image.

Fixed

• Use selected namespace in IntelliJ plugin instead of always using default namespace.

Fixed

• Fix bug where VS Code extension would crash on startup due to new configuration values not being the correct type.
• Unset DYLD_INSERT_LIBRARIES/LD_PRELOAD when creating the agent. Closes [#330].
• Fix NullPointerException in IntelliJ Extension. Closes [#335].
• FIx dylib/so paths for the IntelliJ Extension. Closes [#337].

Added

• Add more configuration values to the VS Code extension.
• Warning when using remote tcp without remote DNS (can cause ipv6/v4 issues). Closes #327

Fixed

• VS Code needed restart to apply kubectl config/context change. Closes 316.
• Fixed DNS feature causing crash on macOS on invalid DNS name due to mismatch of return codes. #321.
• Fixed DNS feature not using impersonated container namespace, resulting with incorrect resolved DNS names.
• mirrord-agent: Use IndexAllocator to properly generate ConnectionIds for the tcp outgoing feature.
• tests: Fix outgoing and DNS tests that were passing invalid flags to mirrord.
• Go Hooks - use global ENABLED_FILE_OPS
• Support macOS with apple chip in the IntelliJ plugin. Closes #337.

New Contributors

• @t4lz made their first contribution in https://github.com/metalbear-co/mirrord/pull/313
• @IsaacCloos made their first contribution in https://github.com/metalbear-co/mirrord/pull/318

Added

• New feature: mirrord now supports TCP traffic stealing instead of mirroring. You can enable it by passing --tcp-steal flag to cli.

Fixed

• mirrord-layer: Go environment variables crash - run Go env setup in a different stack (should fix #292)

Changed

• mirrord-layer: Add #![feature(let_chains)] to lib.rs to support new compiler version.

Fixed

• CI:Release - Fix typo that broke the build

Added

• New feature, tcp outgoing traffic. It's now possible to make requests to a remote host from the staging environment context. You can enable this feature setting the MIRRORD_TCP_OUTGOING variable to true, or using the -o option in mirrord-cli.
• mirrord-cli add login command for logging in to metalbear-cloud
• CI:Release - Provide zip and sha256 sums

Fixed

• Environment variables feature on Golang programs. Issue #292 closed in #299

Fixed

• CI - set typescript version at 4.7.4 to fix broken release action

Added

• Support for Golang fileops
• IntelliJ Extension for mirrord

Changed

• mirrord-layer: Added common Result type to to reduce boilerplate, removed dependency of anyhow crate.
• mirrord-layer: Split LayerError into LayerError and HookError to distinguish between errors that can be handled by the layer and errors that can be handled by the hook. (no more requiring libc errno for each error!). Closes #247

Fixed

• CI - remove usage of ubuntu-18.04 machines (deprecated)

Added

• E2E - add basic env tests for bash scripts

Fixed

• mirrord-agent - Update pcap library, hopefully will fix dropped packets (syn sometimes missed in e2e).
• mirrord-agent/layer - Sometimes layer tries to connect to agent before it finsihed loading, even though pod is running. Added watching the log stream for a "ready" log message before attempting to connect.

Changed

• E2E - describe all pods on failure and add file name to print of logs.
• E2E - print timestamp of stdout/stderr of TestProcess.
• E2E - Don't delete pod/service on failure, instead leave them for debugging.
• mirrord-agent - Don't use tokio::spawn for spawning sniffer (or any other namespace changing task) to avoid namespace-clashing/undefined behavior. Possibly fixing bugs.
• Change the version check on the VS Code extension to happen when mirrord is enabled rather than when the IDE starts up.

Added

• mirrord-layer: You can now pass MIRRORD_AGENT_COMMUNICATION_TIMEOUT as environment variable to control agent timeout.
• Expand file system operations with access and faccessat hooks for absolute paths

Fixed

• Ephemeral Containers didn't wait for the right condition, leading to timeouts in many cases.
• mirrord-layer: Wait for the correct condition in job creation, resolving startup/timeout issues.
• mirrord-layer: Add a sleep on closing local socket after receiving close to let local application respond before closing.
• mirrord-layer: Fix DNS issue where ai_addr would not live long enough (breaking the remote DNS feature).

Changed

• Removed unused dependencies from mirrord-layer/Cargo.toml. (Closes #220)
• reduce e2e flakiness (add message sent on tcp listen subscription, wait for that message)
• reduce e2e flakiness - increase timeout time
• mirrord-layer - increase agent creation timeout (to reduce e2e flakiness on macOS)
• E2E - Don't do file stuff on http traffic to reduce flakiness (doesn't add any coverage value..)
• mirrord-layer - Change tcp mirror tunnel select to be biased so it flushes all data before closing it (better testing, reduces e2e flakiness)
• E2E - unify resolve_node_host for linux and macOS with support for wsl provided Docker & Kubernetes
• E2E - add trace for tests to have paramaterized arguments printed
• mirrord-agent - add debug print of args to identify runs
• E2E - remove double --extract-path parameter in tests
• E2E - macOS colima start with 3 cores and 8GB of RAM.
• E2E - Increase agent communication timeout to reduce flakiness.
• mirrord-layer - add DetourGuard to prevent unwanted calls to detours from our code.
• mirrord-layer - extract reused detours to seperate logic functions
• E2E - macOS run only sanity http mirror traffic with Python

New Contributors

• @DmitryDodzin made their first contribution in https://github.com/metalbear-co/mirrord/pull/244
• @neimsaci made their first contribution in https://github.com/metalbear-co/mirrord/pull/245
• @jim4067 made their first contribution in https://github.com/metalbear-co/mirrord/pull/213

Added

• Add a flag for the agent, --ephemeral-container, to correctly refer to the filesystem i.e. refer to root path as /proc/1/root when the flag is on, otherwise /.

Changed

• Assign a random port number instead of 61337. (Reason: A forking process creates multiple agents sending traffic on the same port, causing addrinuse error.)
• mirrord-layer/socket now uses socket2::SockAddr to comply with Rust's new IP format.

Fixed

• Fix filesystem tests to only run if the default path exists.
• Fix extension not running due to the node_modules directory not being packaged.

Added

• New feature, remote DNS resolving.
  It is now possible to use the remote's addrinfo by setting the MIRRORD_REMOTE_DNS variable to
  true, or using the -d option in mirrord-cli.
• New feature, Ephemeral Containers.
  Use Kubernetes beta feature Ephemeral Containers to mirror traffic with the --ephemeral-container flag.
• E2E tests on macos for Golang using the Gin framework.

Changed

• Refactored mirrord-layer/socket into a module structure similar to mirrord-layer/file.
• Refactored the error part of the many Result<Response, ResponseError>.
• Refactored file related functions, created FileHandler and improved structure.
• Refactored error handling in mirrord-layer.
• E2E: Collect minikube logs and fix collecting container logs
• E2E: macOS use colima instead of minikube.
• Refactored mirrord-layer/lib.rs - no more passing many arguments! :)

Fixed

• Handle unwraps in fileops to gracefully exit and enable python fileops tests.
• Changed addrinfo to VecDeque - fixes a potential bug (loss of order)

Added

• mirrord-cli exec subcommand accepts --extract-path argument to set the directory to extract the library to. Used for tests mainly.
• mirrord-layer provides MIRRORD_IMPERSONATED_CONTAINER_NAME environment variable to specify container name to impersonate. mirrord-cli accepts argument to set variable.
• vscode-ext provides quick-select for setting MIRRORD_IMPERSONATED_CONTAINER_NAME

Changed

• Refactor e2e, enable only Node HTTP mirroring test.
• E2E: add macOS to E2E, support using minikube by env var.
• E2E: Skip loading to docker before loading to minikube (load directly to minikube..)
• layer: Environment variables now load before process starts, no more race conditions.

Fixed

• Support connections that start with tcp flags in addition to Syn (on macOS CI we saw CWR + NS)
• fcntl error on macOS #184 by a workaround.

Added

• mirrord-cli exec subcommand accepts --extract-path argument to set the directory to extract the library to. Used for tests mainly.
• mirrord-layer provides MIRRORD_IMPERSONATED_CONTAINER_NAME environment variable to specify container name to impersonate. mirrord-cli accepts argument to set variable.
• vscode-ext provides quick-select for setting MIRRORD_IMPERSONATED_CONTAINER_NAME

Changed

• Refactor e2e, enable only Node HTTP mirroring test.
• E2E: add macOS to E2E, support using minikube by env var.
• E2E: Skip loading to docker before loading to minikube (load directly to minikube..)
• layer: Environment variables now load before process starts, no more race conditions.

Fixed

• Support connections that start with tcp flags in addition to Syn (on macOS CI we saw CWR + NS)
• fcntl error on macOS #184 by a workaround.

Changed

• Refactor(agent) - change FileManager to be per peer, thus removing the need of it being in a different task, moving the handling to the peer logic, change structure of peer handling to a struct.
• Don't fail environment variable request if none exists.
• E2E: Don't assert jobs and pods length, to allow better debugging and less flakiness.
• Refactor(agent) - Main loop doesn't pass messages around but instead spawned peers interact directly with tcp sniffer. Renamed Peer -> Client and ClientID.
• Add context to agent/job creation errors (Fixes #112)
• Add context to stream creation error (Fixes #110)
• Change E2E to use real app, closes #149

What's Changed

• Faster ci by @aviramha in https://github.com/metalbear-co/mirrord/pull/143
• Add Docker cache to Docker build-push action to reduce build duration. by @aviramha in https://github.com/metalbear-co/mirrord/pull/145
• Refactor traffic by @aviramha in https://github.com/metalbear-co/mirrord/pull/113
• Fix clap deprecations. by @meowjesty in https://github.com/metalbear-co/mirrord/pull/146
• Fix universal binary for macOS by @aviramha in https://github.com/metalbear-co/mirrord/pull/147
• Refactor: Change protocol + mirrord-layer to split messages into modu… by @aviramha in https://github.com/metalbear-co/mirrord/pull/150
• Better logging for Kube pod failures by @thedanvail in https://github.com/metalbear-co/mirrord/pull/144
• Add CLI arg for Agent TTL by @thedanvail in https://github.com/metalbear-co/mirrord/pull/156
• CI Improvements by @aviramha in https://github.com/metalbear-co/mirrord/pull/157
• Handle openat/open relative paths by @infiniteregrets in https://github.com/metalbear-co/mirrord/pull/151
• Fix once cell renamings by @infiniteregrets in https://github.com/metalbear-co/mirrord/pull/159
• Enable "blocking" feature for reqwest by @infiniteregrets in https://github.com/metalbear-co/mirrord/pull/161
• Override remote env #25 by @meowjesty in https://github.com/metalbear-co/mirrord/pull/134
• 2.3.0 by @eyalb181 in https://github.com/metalbear-co/mirrord/pull/163

Full Changelog: https://github.com/metalbear-co/mirrord/compare/2.2.1...2.3.0

What's Changed

• E2E small improvements, removing sleeps by @aviramha in https://github.com/metalbear-co/mirrord/pull/130
• Create macOS universal2 binaries by @messense in https://github.com/metalbear-co/mirrord/pull/140
• 2.2.1 by @eyalb181 in https://github.com/metalbear-co/mirrord/pull/141

New Contributors

• @messense made their first contribution in https://github.com/metalbear-co/mirrord/pull/140

Full Changelog: https://github.com/metalbear-co/mirrord/compare/2.2.0...2.2.1