Bot releases are visible (Hide)
Support for docker-compose
v2 when using "standard" docker daemon. This is due to that docker-compose v2 do not support -H (uses docker context instead).
Published by mariotoffia almost 3 years ago
Fixed CVE-2021-39208 by upgrading SharpCompress.
This requires to drop the support for netstandard1.6 since SharpCompress did drop it.
Please file an Issue if any problems occur due to this!
Cheers,
Mario :)
Published by mariotoffia about 3 years ago
DeleteIfExists(bool removeVolumes = true, bool force = false, string removeLink = null)
It ensures that a container is deleted before creating. Hence the inverse ReuseIfExists()
.
var name = Guid.NewGuid().ToString();
var container = Fd.UseContainer()
.UseImage("postgres:9.6-alpine")
.WithName($"name-{name}")
.Build();
var id = container.Id;
using (var c = Fd
.UseContainer()
.DeleteIfExists()
.UseImage("postgres:9.6-alpine")
.WithName($"name-{name}")
.Build())
{
// Ids should not be equal - since deleted and then created.
AreNotEqual(id, c.Id);
}
This exposes the DateTime
when the container was created.
It is now possible to do UseProjectDirectory()
in the CompositeBuilder
to specify a project directory. This was implemented in PR #217 and brought up in Issue #216.
Added a Example
solution to start showing some simple solutions instead of, possibly, complex unit test to understand how to use this library.
Simplified Trace and support for routing to any logging framework. For example, like this.
Fixed bug: On macOS, tests running the envtest.sh
script would fail with a permission denied exception because the script is not exectuable
Allow to PublishAllPorts
instead of explicit portmappings as a shorthand. Added ExposeAllPorts
.
Hide real exeption when throwOnError
in a container export is fixed.
Fixes many typos
Added a Ductus.FluentDocker.XUnit NuGet package to be used when doing XUnit testing instead of mstest.
Published by mariotoffia over 3 years ago
This is a fix for the CVE-2018-8292. See GitHUB Advisory for more information.
You're strongly advised to update to this release!
Published by mariotoffia over 3 years ago
This release fixes the COPY
command in the docker file builder that did not double quote filenames in the JSON array.
Published by mariotoffia over 3 years ago
Added a limited support to use the FluentAPI to talk to a remote docker daemon without using docker-machine. This is done either by manually creating a instance of a DockerHostService
or use FromUri
on HostBuilder
.
using(var container = Fd.UseHost().
FromUri(Settings.DockerUri, isWindowsHost: true).
UseContainer().
Build())
{
}
The above sample connects to a custom DockerUri
from a setting and is a windows container docker daemon.
FromUri
that uses a DockerUri
to create a IHostService
. This uri is arbitrary. It also support other properties (see below). public HostBuilder FromUri(
DockerUri uri,
string name = null,
bool isNative = true,
bool stopWhenDisposed = false,
bool isWindowsHost = false,
string certificatePath = null) {/*...*/}
It will use "sensible" defaults on all parameters. Most of the case the uri is sufficient. For example if not providing the certificatePath it will try to get it from the environment DOCKER_CERT_PATH. If not found in the environment, it will default to none.
UseHost
that takes a instantiated IHostService
implementation.Published by mariotoffia over 3 years ago
This release just adds a single functionality - Ability to provide with a custome endpoint resolver on a ContainerBuilder
. This solves issue #107.
If the resolver returns null
or is null on the container service, it will use the default mechanism to resolve the IP endpoint.
It can be used in the following manner.
using (
var container =
Fd.UseContainer()
.UseImage("postgres:9.6-alpine")
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.ExposePort(5432)
.UseCustomResolver((
ports, portAndProto, dockerUri) =>
{
if (null == ports || string.IsNullOrEmpty(portAndProto))
return null;
if (!ports.TryGetValue(portAndProto, out var endpoints))
return null;
if (null == endpoints || endpoints.Length == 0)
return null;
if (CommandExtensions.IsNative())
return endpoints[0];
if (CommandExtensions.IsEmulatedNative())
return CommandExtensions.IsDockerDnsAvailable()
? new IPEndPoint(CommandExtensions.EmulatedNativeAddress(), endpoints[0].Port)
: new IPEndPoint(IPAddress.Loopback, endpoints[0].Port);
if (Equals(endpoints[0].Address, IPAddress.Any) && null != dockerUri)
return new IPEndPoint(IPAddress.Parse(dockerUri.Host), endpoints[0].Port);
return endpoints[0];
})
.WaitForPort("5432/tcp", 30000 /*30s*/)
.Build()
.Start())
{
var state = container.GetConfiguration(true/*force*/).State.ToServiceState();
Assert.AreEqual(ServiceRunningState.Running, state);
}
Published by mariotoffia over 3 years ago
This release adds a few features and a handful of bugfixes.
This Release adds a few enhancements to the FileBuilder
to support all dockerfile
commands an fixes a few bugs .
This adds the following commands:
Modified:
Issues: #177 #178 and #179 via PR #180
It is now possible, in ImageBuilder
to do From(string imageAndTag, string asName)
and hence render a FROM image:label AS myimage.
Added simple support to provide an URL instead of a local filepath when using FileBuilder
. It will download the resource and add it to a COPY instruction.
var dockerfile = Fd.Dockerfile()
.UseParent("node:12.18.1")
.Environment("NODE_ENV=production")
.Run("npm install --production")
.Copy(
"https://www.my.com/path/myresource.js",
"/server.js")
.Copy("Resources/Issue/111/server.py", "/server.py")
.Command("node", "server.js").ToDockerfileString();
The above example, copies the myresource.js from https://www.my.com/path to the docker container root /server.js.
Fd().UseContainer()
.UseImage("kiasaki/alpine-postgres")
.WithEnvironment($"POSTGRES_PASSWORD={PostgresPassword}")
.ExposePort(5432)
.ExposePort(8192,8800,"udp")
.WaitForPort("5432/tcp", 30000 /*30s*/);
The ExposePort(int hostPort, int containerPort, string protocol = "tcp")
defaults to tcp but can accept any protocol. Currently docker only support udp.
Published by mariotoffia over 3 years ago
New Features
Bug FIxes:
Published by mariotoffia about 5 years ago
This is a minor update release with some new functionality and a few bugfixes. Most noably:
Added
WithHostName("my custom hostname")
on the ContainerBuilder
.
Added SHELL and COPY on filebuilder to build a Dockerfile. Therefore it is now possible
to write e.g. this in a FileBuilder
Fd.DefineImage("mariotoffia/issue111").ReuseIfAlreadyExists()
.From("microsoft/windowsservercore:1607")
.Shell("powershell", "-Command", "$ErrorActionPreference = 'Stop';")
.Run("powershell [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; exit 0")
.Run(
"Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))")
.Run("choco feature enable --name=allowGlobalConfirmation")
.Run("choco install python3")
.Copy("Resources/Issue/111/server.py", "C:/")
.ExposePorts(8000)
.Command("python", "server.py")
.Builder().UseContainer().UseImage("mariotoffia/issue111")
.WaitForProcess("python.exe", (long) TimeSpan.FromSeconds(30).TotalMilliseconds)
.Builder()
.Build()
.Start()
Added support in TemplateString
to distinguish from http(s) URLS and filepaths when doing modifications on windows systems.
Added IEngineScope
to allow for using a certain docker engine within a IDisposable
clause. It will switch to the engine choosen
and switch back (if needed) to the old engine upon Dispose()
. This is useful in windows environment where a set of images of both
Linux and Windows may be executed. An example:
using (var scope = Fd.EngineScope(EngineScopeType.Windows))
{
using(var win = Fd...)
{
}
}
Fixed bug where WaitForProcess
did not work for Windows containers.
Altered Behaviour on ContainerServiceRemove
where it will not Stop()
the container unless force = true
Added support for Pause
and Resume
(invoke Start
again) on both IContainerService´ and
ICompositeContainerService`
and therefore allow for a single docker container or all containers to be paused and resumed. For example:
using (
var container =
Fd.UseContainer()
.UseImage("postgres:9.6-alpine")
.ExposePort(40001, 5432)
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.Build()
.Start())
{
AreEqual(ServiceRunningState.Running, container.State);
container.Pause();
AreEqual(ServiceRunningState.Paused, container.State);
var config = container.GetConfiguration(true);
AreEqual(ServiceRunningState.Paused, config.State.ToServiceState());
container.Start();
AreEqual(ServiceRunningState.Running, container.State);
config = container.GetConfiguration(true);
AreEqual(ServiceRunningState.Running, config.State.ToServiceState());
}
Added support for netstandard2.1 and therefore works in netcoreapp3.x
Added ability to keep containers and keeprunning in composite services. Earlier
those where all killed. For example keep containers (but stop) on dispose may look like this
var file = Path.Combine(Directory.GetCurrentDirectory(),
(TemplateString) "Resources/ComposeTests/WordPress/docker-compose.yml");
ICompositeService svc = null;
IContainerService c1 = null;
IContainerService c2 = null;
try
{
svc = Fd
.UseContainer()
.UseCompose()
.FromFile(file)
.RemoveOrphans()
.KeepContainer()
.Build().Start();
c1 = svc.Containers.First();
c2 = svc.Containers.Skip(1).First();
svc.Dispose();
Assert.AreEqual(ServiceRunningState.Stopped, c1.State);
Assert.AreEqual(ServiceRunningState.Stopped, c2.State);
}
finally
{
svc?.Dispose();
c1?.Remove(true);
c2?.Remove(true);
}
Published by mariotoffia about 5 years ago
This release fixes a few bugs and updates the SharpCompress reference since it has a bug where it may allow someone to overwrite and execute outside of the extraction point. This is not affecting FluentDocker since it has full control on whats gets extracted. However, if you use this transient reference for your own unzipping this will fix this problem.
Fixed where container build parameters where sent incorrectly (for a docker build).
When a ICompositeContainerService
is shutdown, the referenced containers, images, and hosts was not unreferenced. Now those will be cleared, and it is illegal to access any of those instances (if stored in a variable) after this.
It is now "Prettier" do to a Exec on a container, instead of
using (var c = Fd.UseContainer().UseImage("<your-image>").Build().Start())
{
c.DockerHost.Execute(c.Id, "command and args", c.Certificates);
}
you now can do this
using (var c = Fd.UseContainer().UseImage("<your-image>").Build().Start())
{
c.Execute("command and args");
}
Fixed but in WaitForPort
where it did not count time spent waiting for the Connect
to fail. Now it will sutract
this value and thus is closer to the set timeout. It is, however, not a exact time since no background thread and
CancellationToken
mechanism is employed. This is by design until someone have a really good use-case to do this
on the millisecond.
Cheers,
Mario
Published by mariotoffia about 5 years ago
It now successfully handles named containers in docker-compose files. This fixes Issue #94.
When e.g. a docker-compose specifices named containers like this.
version: '3.5'
services:
# building up the message queue system (zookeeper + kafka)
zookeeper:
image: wurstmeister/zookeeper
container_name: zookeeper
networks:
- messageBus
ports:
- "2181:2181"
restart: always
kafkaserver:
image: wurstmeister/kafka
container_name: kafka
networks:
- messageBus
ports:
- 9092:9092
- 29092:29092
- 29093:29093
restart: always
environment:
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST_DEMOSRV:PLAINTEXT,PLAINTEXT_LOCALHOST:PLAINTEXT
KAFKA_LISTENERS: PLAINTEXT://:9092,PLAINTEXT_HOST_DEMOSRV://:29092,PLAINTEXT_LOCALHOST://:29093
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST_DEMOSRV://demo01:29092,PLAINTEXT_LOCALHOST://localhost:29093
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
# overrides the default separator ',' for creating topics
KAFKA_CREATE_TOPICS_SEPARATOR: "$$'\n'"
# create topic "Values" that will use log compaction to get rid of older value messages
KAFKA_CREATE_TOPICS: |
Values:1:1:compact --config=min.compaction.lag.ms=250 --config=segment.bytes=536870912 --config=segment.ms=10000 --config=retention.ms=10000 --config=min.cleanable.dirty.ratio=0.25 --config=file.delete.delay.ms=0
Configuration:1:1:delete --config=file.delete.delay.ms=0 --config=retention.bytes=-1 --config=retention.ms=-1
ProjectDefinition:1:1:delete --config=file.delete.delay.ms=0 --config=retention.bytes=-1 --config=retention.ms=-1
SetValues:1:1:delete --config=retention.ms=1000 --config=segment.ms=1000 --config=segment.bytes=268435456 --config=min.cleanable.dirty.ratio=0.1 --config=file.delete.delay.ms=0
Devices:1:1:delete --config=file.delete.delay.ms=0 --config=retention.bytes=-1 --config=retention.ms=-1
ReadHistoryValues:1:1:delete --config=retention.ms=1000 --config=segment.ms=1000 --config=segment.bytes=268435456 --config=min.cleanable.dirty.ratio=0.1 --config=file.delete.delay.ms=0
# Enables/disables the possibility to delete a topic (if set to false, the topic will only be marked for deletion but not deleted actually)
KAFKA_DELETE_TOPIC_ENABLE: "true"
networks:
default:
external:
name: nat
# create network for messaging
messageBus:
name: message-bus
the framework now uses the labels for each container to resolve container name, service and instance id.
var file = Path.Combine(Directory.GetCurrentDirectory(),
(TemplateString) "Resources/ComposeTests/KafkaAndZookeeper/docker-compose.yaml");
using (var svc = Fd.UseContainer()
.UseCompose()
.FromFile(file)
.Build()
.Start())
{
var kafka = svc.Services.OfType<IContainerService>().Single(x => x.Name == "kafka");
var zookeeper = svc.Services.OfType<IContainerService>().Single(x => x.Name == "zookeeper");
Assert.AreEqual("kafkaandzookeeper",kafka.Service);
Assert.AreEqual("kafkaandzookeeper",zookeeper.Service);
Assert.AreEqual("1",kafka.InstanceId);
Assert.AreEqual("1",zookeeper.InstanceId);
}
The above starts the compose files containers and resolves them as their named names and extracts the project name and instance id via Labels on each container.
Cheers,
Mario
Published by mariotoffia about 5 years ago
It is now possible to use WaitForPort
with an optional argument address. If it is not supplied, the old behaviour of doing inspect
on the container and grab the network settings is used. However, if it is supplied it will replace the IP address with the supplied one (still resolve the port).
using (var c = Fd.UseContainer()
.WithName("profiles-smtp-server")
.UseImage("mailhog/mailhog:latest")
.ReuseIfExists()
.ExposePort(5011, 8025)
.ExposePort(1025)
.WaitForPort("8025/tcp", TimeSpan.FromSeconds(30), "127.0.0.1")
.Build())
{
c.Start();
var port = c.ToHostExposedEndpoint("8025/tcp").Port;
var response = $"http://127.0.0.1:{port}/".Wget().Result;
IsTrue(response.IndexOf("<title>MailHog</title>", StringComparison.Ordinal) != -1);
}
The above example will use the 127.0.0.1 as the address (loopback) when communicating with port 5011. This helps to solve the Issue #92.
Cheers,
Mario
Published by mariotoffia about 5 years ago
Hi,
This release is based on enhancements and bugfixes by me and @zentron. @zentron provided with the long exit code and health check support to the library.
It is now possible to use HealthCheck
Fluent API to activate the healthcheck docker mechanism (see sample below).
using (
var container =
Fd.UseContainer()
.UseImage("postgres:latest", force: true)
.HealthCheck("exit")
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.Build()
.Start())
{
var config = container.GetConfiguration(true);
AreEqual(HealthState.Starting, config.State.Health.Status);
}
Bugfix: The exit code is assumed to be a long since windows containers sometime exit with a long (such as Ctrl-C).
Docker machine response parser for inspect fails when e.g. --generic-ip-address=docker
is set (Issue #90). The response parser will try to figure out if it is a IP address or a hostname when parsing and set the response accordingly.
Now it is possible to set ulimit on ContainerCreateParams and on
the fluent API (Issue #80).
using (
var container =
Fd.UseContainer()
.UseImage("postgres:latest", force: true)
.UseUlimit(Ulimit.NoFile,2048, 2048)
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.Build()
.Start())
{
// Container running under number of open file limit of 2048
}
Cheers,
Mario
Published by mariotoffia over 5 years ago
This is a small bug fix release where the service hooks was not invoked when ReuseIfExist
was enabled and the container was reused. Now it will add the hooks to those as well and thus it will drive the State
flow and execute any hooks attached. For example the below example would not invoke the CheckConnection prior this release.
Cheers,
Mario
public void WaitLambdaWithReusedContainerShallGetInvoked()
{
_checkConnectionInvoked = false;
using (var c1 = Fd.UseContainer()
.UseImage("postgres:9.6-alpine")
.WithName("postgres")
.ExposePort(5432)
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.ReuseIfExists()
.WaitForPort("5432/tcp", TimeSpan.FromSeconds(30))
.Build()
.Start())
{
// Make sure to have named container running
var config = c1.GetConfiguration();
AreEqual(ServiceRunningState.Running, c1.State);
using (var c2 = Fd.UseContainer()
.UseImage("postgres:9.6-alpine")
.WithName("postgres")
.ExposePort(5432)
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.ReuseIfExists()
.Wait("", (service, count) => CheckConnection(count, service))
.Build()
.Start())
{
IsTrue(_checkConnectionInvoked,
"Is invoked even if reused container since Start drives the container state eventually to running");
}
}
}
private static int CheckConnection(int count, IContainerService service)
{
_checkConnectionInvoked = true;
if (count > 10) throw new FluentDockerException("Failed to wait for sql server");
var ep = service.ToHostExposedEndpoint("5432/tcp");
var str = $"Server={ep.Address};Port={ep.Port};Userid=postgres;" +
"Password=mysecretpassword;Pooling=false;MinPoolSize=1;" +
"MaxPoolSize=20;Timeout=15;SslMode=Disable;Database=postgres";
try
{
using (var conn = new NpgsqlConnection(str))
{
conn.Open();
return 0;
}
}
catch
{
return 500 /*ms*/;
}
}
Published by mariotoffia over 5 years ago
Feature:
using (
var container =
Fd.UseContainer()
.UseImage("postgres:latest", force: true) // default will be false
.ExposePort(5432)
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.WaitForProcess("postgres", 30000 /*30s*/)
.Build()
.Start())
{
var config = container.GetConfiguration(true);
AreEqual(ServiceRunningState.Running, config.State.ToServiceState());
}
Bugs:
DockerBinaryResolver
Published by mariotoffia over 5 years ago
This is a very small bugfix where static IP in a container did not work using the fluent API due to the fact that docker needs to have --network on the container in order to create it. Now it will properly do so on the first available network added to it. The following proves the point:
using (var nw = Fd.UseNetwork("unit-test-nw")
.UseSubnet("10.18.0.0/16").Build())
{
using (
var container =
Fd.UseContainer()
.WithName("mycontainer")
.UseImage("postgres:9.6-alpine")
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.ExposePort(5432)
.UseNetwork(nw)
.UseIpV4("10.18.0.22")
.WaitForPort("5432/tcp", 30000 /*30s*/)
.Build()
.Start())
{
var ip = container.GetConfiguration().NetworkSettings.Networks["unit-test-nw"].IPAddress;
Assert.AreEqual("10.18.0.22", ip);
}
}
Note that it will use the first network provided in UseNetwork
as the --network parameter in docker create …
. This solves the issue #81.
Cheers,
Mario
Published by mariotoffia over 5 years ago
Fixed bug so Docker-Compose
is allowed to execute on remote host. Also Hyper-V docker host service (docker-machine
) do require elevated process nowadays. A workaround since docker-machine inspect machine-name
do return a valid MachineInfo
data and therefore can calculate the URL and set it to Running. Thus, it is not necessary to elevate the process when doing Hyper-V machines in the DockerHostService
.
It is now possible to set explicit container IP both for V4 and V6. This was added to the ContainerCreateParams
and therefore it is possible to supply those both in create or run command such as
var cmd = _docker.Run("postgres:9.6-alpine", new ContainerCreateParams
{
PortMappings = new[] {"40001:5432"},
Environment = new[] {"POSTGRES_PASSWORD=mysecretpassword"},
Network = "mynetwork",
Ipv4 = "1.1.1.1"
}, _certificates);
var container = cmd.Data;
var insp = _docker.InspectContainer(container, _certificates);
var ip = insp.Data.NetworkSettings.IPAddress;
Assert.AreEqual("1.1.1.1", ip);
The above example will create a new container using network mynetwork and set the static IP of 1.1.1.1. The fluent API would looks like this:
using (var nw = Fd.UseNetwork("mynetwork").Build())
{
using (
var container =
Fd.UseContainer()
.WithName("mycontainer")
.UseImage("postgres:9.6-alpine")
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.ExposePort(5432)
.UseNetwork(nw)
.UseIpV4("1.1.1.1")
.WaitForPort("5432/tcp", 30000 /*30s*/)
.Build()
.Start())
{
var ip = container.GetConfiguration().NetworkSettings.IPAddress;
Assert.AreEqual("1.1.1.1", ip);
}
}
It also contains the beginning of docker stack
support. However, it will be finished in a later version and shall not be relied upon (in any way!).
Cheers,
Mario
Published by mariotoffia almost 6 years ago
It is now possible to log into and out of private repository.
The commands added:
_host.Login("server", "user","pass"); // logs into a docker repo
_host.Logout(); // Self explainatory
The above commands gives the ability login and logout. The FluentAPI also got a simple login mechanism (this will change to allow for per container / image basis attach a remote repository login.
Builder().UseContainer().UseImage("imageName").WithCredentials("server", "user","pass");
Only one is supported per fluent chain for now; if you have several different repositories you need to do separate FluentAPI chains for each private repository (this will change in future).
Published by mariotoffia about 6 years ago
This release targets cleanup when errors do occur. It also includes small features such as privileged containers.
A new static type has been introduced, it is called Fd
and it can be used to build, discover hosts and run single container or multiple containers. This will deprecate new Builder()
and new Hosts()
and will be completely removed in 3.0.0. Switch over to use this Fd
class (in namespace Ductus.FluentDocker
).
When an application terminates with a unhandled exception, .net will not always call finally
, therefore if you count to have containers to be cleaned up within a using
may be up for a surprise. This depends. To be completely sure it is possible to use e.g.
// Use Fd.Run for generic cases and Fd.Composite for compose containers
Fd.Container(c => c.UseContainer()
.UseImage("postgres:9.6-alpine")
.ExposePort(5432)
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.WaitForPort("5432/tcp", TimeSpan.FromSeconds(30)),
svc =>
{
var config = svc.GetConfiguration(); // a running container here
});
This can be useful when you may expect an unhandled exception to be thrown after the container is started and before the container is disposed. However, if you have other configuration than Fail Fast or do have a exception handler, you will not need this!
However, nowadays, if the hooks fails such as WaitForPort
it will make sure to Dispose
the instance and therefore stop and delete any underlying container (if not configured to be persisted). This is a change, earlier they did not invoke Dispose
this was up to the invoker to do such.
using (var svc = Fd
.UseContainer()
.UseCompose()
.FromFile(file)
.RemoveOrphans()
.WaitForHttp("wordpress", "http://localhost:8000/wp-admin/install.php")
.Build().Start())
{
var installPage = await "http://localhost:8000/wp-admin/install.php".Wget(); // running service here
}
Sample Builder usage.
Cheers,
Mario