Connect processes into powerful data pipelines with a simple git-like filesystem interface
APACHE-2.0 License
Bot releases are hidden (Show)
Published by avsm over 5 years ago
CHANGES:
conventional
Published by avsm over 5 years ago
CHANGES:
Published by avsm over 5 years ago
CHANGES:
Published by avsm almost 6 years ago
CHANGES:
pcdata
function in favour of txt
(@avsm)dune-release
(@avsm)Published by samoht over 6 years ago
Published by samoht almost 7 years ago
all: update to latest version of alcotest, conduit, session, ocaml-github,
ocaml-github-hooks and cohttp (#612, @samoht and @djs55)
github: make User.t
abstract (#594, @samoht)
github: turn Webhook.events
into a promise (#598, @samoht)
github: add a Comment
module to model PR and issue comments (#595, @samoht)
github: change PR.owner
to be of type User.t
(#599, @samoht)
github-bridge: add the ability to sync PR's coments (#595, @samoht)
go-client: handle large values when reading / writing in 9db (#292, @simonferquel)
go-client: fix the handling of defaults over upgrade (#605, @djs55)
go-client: improve transaction API (#606, @djs55)
Published by samoht over 7 years ago
The main change in this release is the addition of datakit-client-git
which implements the DataKit API on top of a normal Git repository. This
means that the deployment of DataKit tools is now much simpler as they do
not need a running DataKit server anymore. The client and server packages
have been renamed to make the use of 9p more explicit. Support for more
transport is planned, including gRPC and Cap-n-proto.
Go bindings
List
to list files in snapshots (#578, @ebriney)datakit-server, datakit-client
client/server: split the libraires between core API and 9p transport.
There is now:
datakit-client
: signature for client API + Path librarydatakit-server
: implementation of the VFS on top of Irmindatakit-client-9p
: implementation of the API using 9p as transportdatakit-server-9p
: expose the Irmin VFS as the Datakit API; server-sideThe tests are split as well, so all the client/server tests can be re-used
with a different transport mechanism. (#551, @samoht)
client: add a top-level Datakit_client
module namespace: Datakit_S.CLIENT
becomes Datakit_client.S
and Datakit_path
becomes Datakit_client.Path
(#558, @samoht)
client: remove rename
API calls (#563, @samoht)
datakit-client-9p
datakit-client-9p
is now the new name for the previously named
datakit-client
. That package contains the 9p client bindings to the DataKit
API. More clients to come.
DK.commit
now fails if the commit does not exists (instead ofdatakit-client-git
datakit
Datakit
namespace. ExposeDatakit.Blob
, Datakit.Branch
, Datakit.Hash,
Datakit.Metadata
,Datakit.Path
forming the base types for DataKit stores. Also exposeDatakit.Make
(and Datakit.Make_git
) to build a DataKit store from anIrmin
store (or from a Irmin_git
store). Finally, rename the functorDatakit.VFS
datakit-github
datakit-github-bridge
github bridge: look at the GH token in various places (#577, @samoht)
github bridge: add the ability to monitor default repositories using the CLI
(#577, @samoht)
github bridge: allow to use git://<path>
urls to "connect" to a local Git
repo instead of a 9p DataKit server (#577, @samoht). For instance:
$ datakit-bridge-github -r samoht/test -d git:///tmp/foo --resync 60
will download all the issues and PR into a Git repository /tmp/foo
and will
keep it up-to-date when changes occur either on GitHub (with a full resync
every 60s) or locally by commiting updates in the /tmp/foo
Git repository.
datakit-ci
Published by samoht over 7 years ago
all
Published by samoht over 7 years ago
DataKit has a new home: the Moby Project!
all
docker run datakit/db
to run the database (see README.md)Datakit_conduit
were depending on datakit-server and it wasdatakit-ci
datakit
datakit-github
datakit-bridge-github
/run/secrets/github-jar
(#519, @talex5)datakit-bridge-local-git
Published by samoht over 7 years ago
datakit-local-git (new)
Normally, we use datakit-github to monitor the state of a remote
repository on GitHub and use that as the input to the CI. When getting
started with DataKitCI it is more convenient to be able to monitor a
local Git repository.
datakit-bridge-github (new)
datakit-github
package into 2: datakit-github
datakit-bridge-github
which matchdatakit-bridge-local-git
(#480, @samoht)datakit-github
Datakit_github
, an abstractdatakit-ci
datakit-gitub
to manipulate GitHub metadaa.Target.v
(#430, @avsm)datakit-client
Published by samoht almost 8 years ago
datakit-github
datakit-ci
datakit-github
--resync-interval
option to resync the database regularly.dirty
files to tell the bridge to resync on repo/prsdatakit-github.client
,datakit-github.server
and datakit-github
(#375, @samoht)Snapshot.find
(#376, @samoht)Repo.Map
, Repo.of_string
(#376, @samoht)Satus.compare_id
(#376, @samoht)create
functions by v
to be consistent (#376, @samoht)Index
modules for PRs, refs and build status (#377, @samoht)datakit-client
tcp:foo
tcp://foo
and fd:42
becomes fd://42
(#358, @samoht)datakit-server
--sandbox
argument (#357, @samoht)tcp:foo
tcp://foo
and fd:42
becomes fd://42
(#358, @samoht)The highlight of that release is datakit-ci
: a new library to help
creating new CI pipelines built on top of DataKit.
datakit-ci
datakit-github
datakit
datakit-client
datakit-server
Vfs.Logs
to expose the state of Logs.Src
over 9p. This/debug
and let otherdatakit-server
library, to help adding runtime instrospectiondatakit-github
datakit-client
datakit-server
--listen-http
Split the package in 3: server-side, client-side and GitHub bridge
(#232, @samoht)
datakit-github
datakit-server
Git_unix.read_file_with_read
(#88, @chambart)--eventlog
command-line argument into--log-destination=(stderr|asl|eventlog)
(#115, @djs55)--bare
the default behavior (#185, @samoht)/debug
directory to control log levels (#239, @talex5)datakit-client
Published by samoht over 8 years ago
Bindings to the Github events API to get notification on new pull requests
(#63, @samoht)
while read PR do
echo New PR: $PR
echo pending > /db/github.com/docker/datakit/pr/status/test
...
done < /db/github.com/docker/datakit/pr/updates
Allow to set the description and urls for pull request status (#64, @samoht)
Update VFS to report file metadata (#66, @talex5)
Add stats
to file metadata (#66, @talex5)
Replace "Buffer to small" error with a read of 0 items (#71, @talex5)
Linux reads a directory like like:
Allow renaming files in a transaction (#73, @talex5)
Metadata support (executable files and symlinks):
Don't allow fast-forward to missing commit (#81, @talex5)
Simplify the Vfs.File.Stream
API and use it consistently through the
codebase (#83, @samoht)
Published by samoht over 8 years ago
Fix a memory leak. This was due to a missing free
in the readdir
bindings in the upstream Lwt library, see
https://github.com/ocsigen/lwt/issues/229 (#57 by @talex5 and @samoht)
Fix a regression introduced in 0.2.0 where inode numbers were not stable
anymore, which broke cp
commands (#56 by @talex5)
Enable verbose mode by setting the DATAKIT_VERBOSE environment variable
and turn-on more verbose logging of underlying libraries (#53 by @samoht)
Replace Irmin views with a more efficient representation which does not
keep track of transaction's actions and remove the need for creating temporary
dangling commits (#16, #51 by @talex5)
Backport performance fixes from upstream ocaml-git
(#43, #47 by @talex5)
Minimal integration with the Github PR API using 9p (#42 by @samoht)
~/mnt $ ls github.com/docker/datakit
41 42
~/mnt $ cat github.com/docker/datakit/pr/41/status/default
pending
~/mnt $ echo success > github.com/docker/datakit/pr/41/status/default
Use Astring and Logs (#50, #43 by @talex5 and @samoht)
Published by samoht over 8 years ago
Add support for "Git remotes" in the filesystem (#21, @samoht). Now you can:
~/mnt $ cd remotes
~/mnt/remotes $ mkdir origin
~/mnt/remotes $ echo https://github.com/docker/datakit > origin/url
~/mnt/remotes $ echo master > origin/fetch
~/mnt/remotes $ cat origin/head
4b6557542ec9cc578d5fe09b664110ba3b68e2c2
Remove dependency to gmp (@samoht)
Split the library into 3 parts (@samoht)
Add CI scripts to test Datakit build using Datakit (@talex5)
Restructure the build process (@samoht)
Add a -bare
option for storing local state in a bare Git repo (@talex5)
Updates for latest ocaml-9p API changes (@talex5)
Published by samoht over 8 years ago
Published by samoht over 8 years ago
Initial release. Code extracted from:
docker/pinata 56982b9e744da6f895a0a4682f2e870f526d2ea3