mirrord

Added

• CI: Update homebrew formula on release, refer #484

Changed

• VS Code Extension: change extension to use the target specified in the mirrord config file, if specified, rather than show the pod dropdown

Added

• MIRRORD_AGENT_NETWORK_INTERFACE environment variable/file config to let user control which network interface to use. Workaround for #670.
• mirrord-config: deprecated and unstable tags to MirrordConfg macro for messaging user when using said fields

Changed

• VS Code Extension: change extension to use a mirrord-config file for configuration
• VS Code Extension: use the IDE's telemetry settings to determine if telemetry should be enabled

Changed

• mirrord-layer: Remove unwrap from initialization functions.
• Log level of operation bypassing log from warn to trace (for real this time).
• Perform filesystem operations for paths in /home locally by default (for real this time).

Added

• VS Code Extension: add JSON schema
• Bypass SIP on MacOS on the executed binary, (also via shebang).
  See [#649].
  This does not yet include binaries that are executed by the first binary.

Fixed

• fix markdown job by adding the checkout action

Full Changelog: https://github.com/metalbear-co/mirrord/compare/3.7.3...3.8.0

Fixed

• mirrord-agent: No longer resolves to eth0 by default, now we first try to resolve the appropriate network interface, if this fails then we use eth0 as a last resort. Fixes #670.

Changed

• intelliJ: use custom delve on macos

Fixed

• Release: fix broken docker build step caused by folder restructure

Fixed

• using gcloud auth for kubernetes. (mistakenly loaded layer into it)
• debugging Go on VSCode. We patch to use our own delivered delve.
• Changed layer not to crash when connection is closed by agent. Closed #693.

Changed

• IntelliJ: fallback to using a textfield if listing namespaces fails

Added

• mirrord-config: New mirrord-schema.json file that contains docs and types which should help the user write their mirrord config files. This file has to be manually generated (there is a test to help you remember).

Fixed

• IntelliJ: Fix occurring of small namespace selection window and make mirrord dialogs resizable
• IntelliJ: Fix bug when pressing cancel in mirrord dialog and rerunning the application no mirrord window appears again
• VS Code: Fix crash occurring because it used deprecated env vars.

Changed

• mirrord-config: Take schema feature out of feature flag (now it's always on).
• mirrord-config: Add docs for the user config types.

Added

• mirrord-layer: Allow capturing tracing logs to file and print github issue creation link via MIRRORD_CAPTURE_ERROR_TRACE env variable

Fixed

• Fix vscode artifacts where arm64 package was not released.
• IntelliJ plugin: if namespaces can't be accessed, use the default namespace

Changed

• Add /home to default file exclude list.
• Changed log level of Bypassing operation... from warning to trace.
• IntelliJ settings default to match CLI/VSCode.

Fixed

• Fixed broken release step for VS Code Darwin arm64 version

Fixed

• Fixed breaking vscode release step

Fixed

• Fixed an issue with the release CI

Changed

• Update target file config to have namespace nested inside of target and not a separate target_namespace. See #587 and #667

Added

• aarch64 release binaries (no go support yet, no IntelliJ also).
• mirrord-layer: Add FileFilter that allows the user to include or exclude file paths (with regex support) for file operations.

Changed

• mirrord-layer: Improve error message when user tries to run a program with args without --.
• Add tests for environment variables passed to KubeApi for authentication feature for cli credential fetch
• Remove openssl/libssl dependency, cross compilation is easier now. (It wasn't needed/used)
• mirrord-config: Changed the way fs works: now it supports 2 modes Simple and Advanced, where Simple is similar to the old behavior (enables read-only, read-write, or disable file ops), and Advanced allows the user to specify include and exclude (regexes) filters for FileFilter.
• Lint README and update it for --target flag.
• mirrord-layer: improve error message for invalid targets.

Removed

• --pod-name, --pod-namespace, --impersonated_container_name have been removed in favor of --target, --target-namespace

Fixed

• Env var to ignore ports used by a debugger for intelliJ/VSCode, refer #644

Added

• Add changelog for intelliJ extension, closes #542
• Add filter for changelog to ci.yml
• Telemetry for intelliJ extension.

Changed

• Update intelliJ extension: lint & bump java version to 17.
• Added /Users and /Library to path to ignore for file operations to improve UX on macOS.
• Use same default options as CLI in intelliJ extension.
• Improve UI layout of intelliJ extension.
• Separate tcp and udp outgoing option in intelliJ extension.
• Tighter control of witch environment variables would be passed to the KubeApi when fetching credentials via cli in kube-config. See #637

Fixed

• Lint Changelog and fix level of a "Changed" tag.
• File operations - following symlinks now works as expected. Previously, absolute symlinks lead to use our own path instead of target path. For example, AWS/K8S uses /var/run/.. for service account credentials. In many machines, /var/run is symlink to /run so we were using /run/.. instead of /proc/{target_pid}/root/run.
• Fix not reappearing window after pressing cancel-button in intelliJ extension.

Added

• Telemetries, see TELEMETRY.md for more information.

Changed

• Added timeout for "waiting for pod to be ready..." in mirrord-layer to prevent unresponsive behavior. See #579
• IntelliJ Extension: Default log level to ERROR from DEBUG

Fixed

• Issue with bottlerocket where they use /run/dockershim.sock instead of the default containerd path. Add new path as fallback.

Changed

• Extended support for both -s and -x wildcard matching, now supports PREFIX_*, *_SUFFIX, ect.
• Add to env default ignore JAVA_HOME,HOMEPATH,CLASSPATH,JAVA_EXE as it's usually runtime that you don't want from remote. Possibly fixes issue discussed on Discord (used complained that they had to use absolute path and not relative).
• Add jvm.cfg to default bypass for files.
• Clarify wrong target error message.
• mirrord-layer: Improve error message in connection::handle_error.

Fixed

• Don't ignore passed --pod-namespace argument, closes
  [#605]
• Replace deprecated environment variables in IntelliJ plugin
• Issues with IntelliJ extension when debugging Kotlin applications
• Scrollable list for pods and namespaces for IntelliJ extension, closes [#610]

Deprecated

• --impersonated-container-name and MIRRORD_IMPERSONATED_CONTAINER_NAME are
  deprecated in favor of --target or MIRRORD_IMPERSONATED_TARGET
• --pod-namespace and MIRRORD_AGENT_IMPERSONATED_POD_NAMESPACE are deprecated in
  favor of --target-namespace and MIRRORD_TARGET_NAMESPACE

New Contributors

• @jamowei made their first contribution in https://github.com/metalbear-co/mirrord/pull/612

Changed

• release: VS Code extension release as stable and not pre-release.

Changed

• Update some texts in documentation, READMEs, and extension package descriptions
• IntelliJ version check on enabling instead of on project start. Don't check again after less than 3 minutes.

Fixed

• IntelliJ plugin crashing on run because both include and exclude were being set for env vars.

Added

• pwrite hook (used by dotnet);
• Reuse agent - first process that runs will create the agent and its children will be able to reuse the same one to avoid creating many agents.
• Don't print progress for child processes to avoid confusion.
• Skip istio/linkerd-proxy/init container when mirroring a pod without a specific container name.
• Add "linkerd.io/inject": "disabled" annotation to pod created by mirrord to avoid linkerd auto inject.
• mirrord-layer: support -target deployment/deployment_name/container/container_name flag to run on a specific container.
• /nix/* path is now ignored for file operations to support NixOS.
• Shortcut deploy for deployment in target argument.
• Added the ability to override environment variables in the config file.
• pread hook (used by dotnet);
• mirrord-layer: ignore opening self-binary (temporal SDK calculates the hash of the binary, and it fails because it happens remotely)
• Layer integration tests with more apps (testing with Go only on MacOS because of
  known crash on Linux - [#380]).
  Closes [#472].
• Added progress reporting to the CLI.
• CI: use bors for merging! woohoo.
• Release image for armv7 (Cloud ARM)
• Layer integration test. Tests the layer's loading and hooking in an http mirroring simulation with a flask web app.
  Addresses but does not
  close [#472] (more integration tests still needed).
• mirrord-cli: added a SIP protection check for macos binaries, closes [#412]
• Release CI: add extensions as artifacts, closes [#355]
• Support impersonated deployments, closes [#293]
• Shorter way to select which deployment/pod/container to impersonate through --target or MIRRORD_IMPERSONATED_TARGET, closes [#392]
• mirrord-layer: Support config from file alongside environment variables.
• intellij-ext: Add version check, closes [#289]
• intellij-ext: better support for Windows with WSL.
• Test that verifies that outgoing UDP traffic (only with a bind to non-0 port and a
  call to connect) is successfully intercepted and forwarded.
• mirrord-layer, mirrord-cli: new command line argument/environment variable - MIRRORD_SKIP_PROCESSES to provide a list of comma separated processes to not to load into.
  Closes [#298], [#308]
• release CI: add arm64e to the universal dylib
• intellij-ext: Add support for Goland
• Code sign Apple binaries.
• CD - Update latest tag after release is published.
• New feature: UDP outgoing, mainly for Go DNS but should work for most use cases also!
• E2E: add tests for python's fastapi with uvicorn
• Socket ops - connect: ignore localhost and ports 50000 - 60000 (reserved for debugger)
• Add "*.plist" to IGNORE_REGEX, refer [#350].

Fixed

• Issue #577. Changed non-error logs from error! to trace!.
• Issue #531. We now detect NixOS/Devbox usage and add sh to skipped list.
• Fix IntelliJ Extension artifact - use glob pattern
• Use LabelSelector instead of app=* to select pods from deployments
• Added another protection to not execute in child processes from k8s auth by setting an env flag to avoid loading then removing it after executing the api.
• Fixed unused dependencies issue, closes [#494]
• getaddrinfo now uses trust-dns-resolver when resolving DNS (previously it would do a getaddrinfo call in mirrord-agent that could result in incompatibility between the mirrored pod and the user environments).
• Support clusters running Istio. Closes [#485].
• tcp-steal working with linkerd meshing.
• mirrord-layer should exit when agent disconnects or unable to make initial connection
• release CI: Fix dylib path for dd.
• mirrord-layer: Fix connect returning error when called on UDP sockets and the
  outgoing traffic feature of mirrord is disabled.
• mirrord-agent: Add a tokio::time:timeout to TcpStream::connect, fixes golang issue where sometimes it would get stuck attempting to connect on IPv6.
• intelliJ-ext: Fix CLion crash issue, closes [#317]
• vscode-ext: Support debugging Go, and fix issues with configuring file ops and traffic stealing.
• mirrord-layer: Return errors from agent when connect fails back to the hook (previously we were handling these as errors in layer, so connect had slightly wrong behavior).
• mirrord-layer: instrumenting error when write_detur is called to stdout/stderr
• mirrord-layer: workaround for presented server name type wasn't supported error when Kubernetes server has IP for CN in certificate. [#388]
• Outgoing UDP test with node. Closes [#323]
• Fix crash in VS Code extension happening because the MIRRORD_OVERRIDE_ENV_VARS_INCLUDE and MIRRORD_OVERRIDE_ENV_VARS_EXCLUDE vars being populated with empty values (rather than not being populated at all).Closes [#413].
• Add exception to gradle when dylib/so file is not found. Closes [#345]
• mirrord-layer: Return errors from agent when connect fails back to the hook (previously we were handling these as errors in layer, so connect had slightly wrong behavior).
• Fix Environment parsing error when value contained '='
  Closes [#387].
• Fix bug in outgoing traffic with multiple requests in quick succession.
  Closes [#331].
• Add missing dependency breaking the VS Code release.
• mirrord-layer: User-friendly error for invalid kubernetes api certificate
• mirrord-cli: Add random prefix to the generated shared lib to prevent Bus Error/EXC_BAD_ACCESS
• Support for Go 1.19>= syscall hooking
• Fix Python debugger crash in VS Code Extension. Closes [#350].

Changed

• Agent pod definition now has requests specifications to avoid being defaulted to high values. See #579.
• Change VSCode extension configuration to have file ops, outgoing traffic, DNS, and environment variables turned on by default.
• update intelliJ extension: toggles + panel for include/exclude env vars
• Exclude internal configuration fields from generated schema.
• Print exit message when terminating application due to an unhandled error in the layer.
• mirrord-layer: refactored pod_api.rs to be more maintainble.
• Use kube config namespace by default.
• mirrord-layer: Ignore EAFNOSUPPORT error reporting (valid scenario).
• Don't report InProgress io error as error (log as info)
• mirrord-layer: Added some dotnet files to IGNORE_FILES regex set;
• mirrord-layer: Added the Detour type for use in the ops modules instead of HookResult. This type supports returning a Bypass to avoid manually checking if a hook actually failed or if we should just bypass it;
• mirrord-protocol: Reduce duplicated types around read operation;
• Layer integration tests for more apps. Closes
  [#472].
• Rename http mirroring tests from integration to http_mirroring since there are
  now also integration tests in other files.
• Delete useless e2e_macos CI job.
• Integration tests also display test process output (with mirrord logs) when they
  time out.
• CI: mirrord-layer UT and integration run in same job.
• .devcontainer: Added missing dependencies and also kind for running e2e tests.
• Replaced pcap dependency with our own rawsocket to make cross compiling faster and easier.
• Remote operations that fail logged on info level instead of error because having a file not found, connection failed, etc can be part of a valid successful flow.
• mirrord-layer: When handling an outgoing connection to localhost, check first if it's a socket we intercept/mirror, then just let it connect normally.
• mirrord-layer: removed tracing::instrument from *_detour functions.
• Ignore http tests because they are unstable, and they block the CI.
• Bundle arm64 binary into the universal binary for MacOS.
• mirrord-layer: Remove check for ignored IP (localhost) from connect.
• mirrord-layer: Refactor connect function to be less bloated.
• .dockerignore now ignores more useless files (reduces mirrord-agent image build time, and size).
• mirrord-agent: Use tracing::instrument for the outgoing traffic feature.
• mirrord-agent: IndexAllocator now uses ConnectionId for outgoing traffic feature.
• mirrord-layer: Remove tracing::instrument from go_env::goenvs_unix_detour.
• mirrord-layer: Log to info instead of error when failing to write to local tunneled streams.
• mirrord-layer: Use tracing::instrument to improve logs.
• Changed agent namespace to default to the pod namespace.
  Closes [#404].
• In go-e2e test, call os.Exit instead fo sending SIGINT to the process.
• Install script now downloads latest tag instead of main branch to avoid downtime on installs.
• Change all functionality (incoming traffic mirroring, remote DNS outgoing traffic, environment variables, file reads) to be enabled by default. Note that flags now disable functionality

Deprecated

• --pod-name or MIRRORD_AGENT_IMPERSONATED_POD_NAME is deprecated in favor of --target or MIRRORD_IMPERSONATED_TARGET

3.0.22-alpha

Changed

• Exclude internal configuration fields from generated schema.

Fixed

• Issue #531. We now detect NixOS/Devbox usage and add sh to skipped list.

Full Changelog: https://github.com/metalbear-co/mirrord/compare/3.0.21-alpha...3.0.22-alpha