arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

arcadia-v0.2.0

Welcome to this new release!

Images built for this release:

• kubeagi/arcadia:v0.2.0
• kubeagi/data-processing:v0.2.0

Breaking Changes:

None

Feature summary 🚀 🚀 🚀

For dataprocessing:

1. Add a new datasource Web and able to download,extract,process online url resources to enhanced QA
2. Deduplicate QA by similarity comparison on Q's embeddings

For model:

1. Support more models: mistral,qwen7b,qwen

For LLM application and agents:

1. Application/Agent evaluation with the help of ragas_once(a CLI of ragas) in following metrics: context_precision,context_recall,context_relevancy,answer_relevancy,answer_similarity,answer_correctness,faithfulness
2. Add a new CRD Agent to provides agent capabilities
3. Add two new tools: bing_search, weather query
4. Able to process documents(pdf,txts) during chat: generate embeddings and summary with a MapReduceChain
5. Add a new Chain ApiChain which call a open api during chat
6. Able to publish applications to gpt store

Changelog

New Features

• 0ce6076a980cab2b6bfc9de7d759c85c29435ed4: feat: Add 1st version of agent support with a weather query tool (@nkwangleiGIT)
• 8514d7baf524e85084f5c894aacb2317594a4f07: feat: Add api to support copying existing rag (@dayuy)
• 07a62dea1f0c33da91716b6463994e4f4d2ae070: feat: able to customize worker' images (@0xff-dev)
• 149fb59eaa63044a509dac7fae1f3ee3fa699190: feat: able to generate embeddings and summarizations for documents in a conversation (@bjwswang)
• d3b30395db8d49e480c9038551c2e228e5d25a66: feat: able to set/update models in LLM and Embedder (@bjwswang)
• c519fe5dd28d71307b23b011626c1430630755dd: feat: able to store chat document (@bjwswang)
• 540607d153e0aadf557eec61899fca6d598d9808: feat: able to store chat document (@bjwswang)
• 27f9ef0183c5f3babe0e8bbb624faaf92e0fdf15: feat: able to use Web datasource in versioneddataset (@bjwswang)
• e0cd2350f7499ad37cb6a75e38afad38ab1b10ed: feat: able to use openai as the embedder in knowledgebase (@bjwswang)
• 46f39ede1d6c6dbdae44844d946d90fff962298c: feat: add README.md for arcadia chart (@Lanture1064)
• 0703063382a85596264d451968ae5e7677fa2c68: feat: add a command to download a file (@0xff-dev)
• 98923c371df3f40356669e5bc39c5a493ac1be4e: feat: add a controller implementation of rag (@0xff-dev)
• dc10c0039eeff485858f05b0abf21baa5cda656a: feat: add a new api to get rag evaluation report (@0xff-dev)
• 4b5e25fcbc50a4ab2a30a602d187a1a71ca13874: feat: add a new column latency in rag eval dataset row (@0xff-dev)
• 89a65ae24cf21886604810e8971d5ea83cb06080: feat: add a new mapreducechain in appruntime (@bjwswang)
• 181b3df28b6b507d087d101d7842ae0837d43bbd: feat: add api and doc support to return to the download link (@xxxxibo)
• 0d6e83d5fc4bd0f11181d98cc22be345146caa41: feat: add apichain (@Abirdcfly)
• 22ba5c4cd158000ef9dcb69da7f9dbb0d453b4b7: feat: add bingsearch tools (@Abirdcfly)
• 5ec0900e8c8da9640747d596289e534a5f30be1d: feat: add chat latency and api: generate prompt starters (@Abirdcfly)
• f7b12e2be1690e71570baa6427e4f4c261e44c09: feat: add creator for rag (@0xff-dev)
• ba3936997ca89ea2b67334c4f3738954aea6ec53: feat: add gpt list (@Abirdcfly)
• e3f036f74423275b5b532d0c7f807f9d15306a47: feat: add graphql definition of rag (@0xff-dev)
• 59c59c9311bdd442b0585e6debc9db67f339706a: feat: add maxTokens in apiserver (@Abirdcfly)
• fc79c1a73e92dab5936f713f8f1835113ab29623: feat: add more info to model including MaxContextLength,HuggingfaceRepo,ModelscopeRepo (@bjwswang)
• 51a709bf7636958fd6cda32b49bb326b2579c101: feat: add new datasource Web (@bjwswang)
• ac9e12114c3122e77ad87e446de599488c9d8f16: feat: add rag custom resource definition (@0xff-dev)
• 008ad775541868f765e0132b2eafa8bcfb8eb2d9: feat: add ragas_once tool (@Lanture1064)
• 146af318794d8c5140050d78f129a75b882aee3e: feat: add reocrd costtime (@0xff-dev)
• 736803e7f89933e26821f75b4baadd26b9e7ae75: feat: add some display config for app (@Abirdcfly)
• 9a4932e807ee889746387954e092ad6b239077d0: feat: added watch on Embedder and VectorStore for KnowledgeBase reconciliation (@Abirdcfly)
• 6eeceee237b56ce06efec00fdbad525d25b9d6ab: feat: apiserver authentication support different groups (@0xff-dev)
• 09254c935228836d35bab43d487a22ae9c47c2ca: feat: arctl support merge files (@0xff-dev)
• 83e72f51c9d223db4c11df62268085084e920275: feat: bingsearch add more config (@Abirdcfly)
• 36cd8dbf8b3b4ad4fb5d0e324f86d19901833369: feat: change maxLength default value to 2048 (@Abirdcfly)
• 3dd25d3e965244051f75f0363acc99f1fe9c6fd2: feat: chat info store to pg (@Abirdcfly)
• 7c6ef7f41afae85f929c8348abf81e4e1d477821: feat: generate evaluation test dataset (@bjwswang)
• 2d22e20c14e5c2c118d32d7db8558e500f60cb5d: feat: graphql add api to create rag (@0xff-dev)
• 534b4027175de8abc4670e5291a4d4cfd1817425: feat: graphql: add api to update rag (@dayuy)
• 199d0b08a2db5697014cab01a242fb11f3eb4ff3: feat: handle pod state Waiting in worker (@bjwswang)
• b01bbdb3672ccd2232157717972fe128c9387c76: feat: knowledgebase support more granular update (@Abirdcfly)
• 4f4b17dc9392a952c16e3ec94a46410d5d1855eb: feat: namespace controller creates the rabc needed for the job (@0xff-dev)
• dc88d12727885fbe8aa4bd01ea67f2757de134b1: feat: release v0.2.0 (@bjwswang)
• c0ff0914df9a59f2687322e89812a67d83e86e22: feat: result & summary CSV file output (@Lanture1064)
• 804542dd2d5e78e00db0398d2a8e7a5bff0fd353: feat: return users all conversation (@Abirdcfly)
• cabcb458c00a5041aacdbb261be8de0a1479410a: feat: set to storage as a non-required field (@0xff-dev)
• 0dff6858a86d94ddc2972e446af414792c2cdeef: feat: support for downloading data from versioneddatasets (@0xff-dev)
• 237777524c354c6885b19c92836fac6d9975f781: feat: support streaming and chain of agent (@nkwangleiGIT)
• f234364a9e364ac55e72c52a74701581f44dce99: feat: support streaming in chat docs (@bjwswang)
• 461309d6086202a8889ce3723f12b37820d1eb46: feat: unify the implementation of list apis (@0xff-dev)
• 21f9aaa4e4a0d10dc9fa4ab862d24d4bb2c402e1: feat: when a task fails, display the failure time (@0xff-dev)
• a74b3d209de8e4c2b36ebf8f8c1f3484e235c20f: feat:QA remove duplicate (@)
• 961797113e7576798841a6c5aad0dba039e0435c: feat:Real-time data access (@)
• fe1190db8727bf507b23ea795df236dfa683c1f2: feat:Splitting website data (@wangxinbiao)
• af1ee1b202abde544dd37c8eeac9f0d87719fbe8: feat:able to use openai in data-processing (@wangxinbiao)
• af8f14c26247c28776d0167f4a730041cf0b901c: feat:add dataprocessing task retry functionality (@wangxinbiao)
• d79d56befa4ac1de506bac759af705ef7966c7a5: feat:add fields dataProcessMs and syncMsg for the data processing information and file synchronisation information respectively (@xxxxibo)
• cd8550fcda8f62b23d829a0021b6ee233958ae12: feat:deduplicate QA (@wangxinbiao)
• 77f512973a1fe60d370c9965d08244f03238309c: feat:details return deduplication configuration information (@wangxinbiao)
• 97ce84c1df8c5b1a3d4bfdb5f2ba608dfa1d64f6: feat:support downloading files to specific directories (@xxxxibo)
• 49ed6bcd457335f34534ecddd1517ef705aae1f6: feat:when the task fails, update the 'message' information in the dataset for cr. (@wangxinbiao)

Bug Fixes

• 7156ee803278289965c573138cbe66d297caecb6: fix(application): add isReady check for all nodes in application (@Abirdcfly)
• 1ae5ff1600393d0fa425822bb10d2ff75d9c0d24: fix(appruntime): Add Cleanup() function to Node interface (@Abirdcfly)
• a99b1b5c8e96491c43defb6d91103a8a3f3a320a: fix: Failed to create Versioneddataset with the inherited historical version (@dayuy)
• d00c5bcd0d81e825f10d38fab53fbd23cf4fd4b8: fix: able to use openai-compatible llms/embeddings in ragas_once (@bjwswang)
• 16dc3d5469a30a853bed4649bcf43f8c3d11607b: fix: adapting ragas_once parameter changes (@0xff-dev)
• e07201e08c30a884291640c7255b370842fc51f2: fix: add displayName in graphql TypedObjectReference (@bjwswang)
• a2712ab786d0215e28df6691d31e7329b62ae7b8: fix: add empty history for evaluation run (@Abirdcfly)
• 82309518e185769d746f688c0b9a9776e082985f: fix: add messageID for stream mode (@Abirdcfly)
• 5bf9bfca53609d97f111ea62905a20c867593b12: fix: add new filed embeddderType in apiserver's knowledgebase model (@bjwswang)
• 8b3111270522506850d6c0ca9095bccda5540d5a: fix: add permissions for arctl (@0xff-dev)
• 3e124212c1e757e633fd4c45cbf66cbe742b8a3f: fix: apiserver needs access to storageclass (@0xff-dev)
• 51b48561ff1adaf38d16fe7bd003f154c8ac9abb: fix: application get more fields (@Abirdcfly)
• 875d096bb95bc94dc739cbf55585c21d638ced59: fix: avoid the pointer to loop variables (@xxxxibo)
• ba85408cf8dc1b62f86712f9c076d29dc7321cac: fix: avoid to update resource status with explictly status check (@bjwswang)
• bcf9cfe9608b6b28a0fd5a917e0f5446e3f233ca: fix: datasource and dataset are using the wrong bucket in command arctl (@0xff-dev)
• 9a70257738402ca0328622eec4cd68e8c5bb1309: fix: delete the rag and clean up the minio files at the same time (@0xff-dev)
• 0fc1427349a0be44e9ef13c1dd55124dbc8ea245: fix: doc score need update (@Abirdcfly)
• 602efc09340aaa5bab550697ca559608b328063c: fix: gpt hot use storage interface (@Abirdcfly)
• 2699be2f13188bda45a798b18b40afd4d213b318: fix: handle some errors in chat when using stream mode (@Abirdcfly)
• d30a3ceaa4368e6dd54fc51bf8e2726fb1886ddb: fix: init a start time when chat documents (@bjwswang)
• 48482ebb81b41dbda6c51d286b212d86e0a37d2f: fix: langchaingo upstream update (@Abirdcfly)
• b7629468c39efb46f95226ab0f2ba9a1c1d8ab03: fix: pass chain call options to chat_docs (@bjwswang)
• ee6160184111110fd6f49bc7e20902de3416079e: fix: pgvector return nothing when embedding table has vector with different dimensions (@Abirdcfly)
• ca249e746c5a378281e5ef80abad73602f3a2a5c: fix: prompt is using the wrong group (@0xff-dev)
• fc94bf127011dc0c10bb6ec4a693a8e52e9ad543: fix: prompt-starter api cant get data when use knowledgebase (@Abirdcfly)
• 1e7b2b8798faaa953f0eaeade2ed9c31e15b0e25: fix: qachain set history will get error (@Abirdcfly)
• 9149421ebef40ae5a3e24268275dbab75aa49a27: fix: recover data-processing image build action (@bjwswang)
• d2b91b667af456a10833dfef1b6775f84afa69ed: fix: remove ticker check in chat docs (@bjwswang)
• 2248446af68c4a9a1e9d46548d8ccb7f505e294a: fix: replce comma with underline in woker's model label (@bjwswang)
• c0e537b69a6378aa37eab31a9d7c6ee070a84360: fix: set model in QAProviderZhiPuAIOnline (@bjwswang)
• b31d71f31fff00715db863d2de1ace374559d1d7: fix: shoud delete worker label with DeepCopied object (@bjwswang)
• d5775c4aca4f33573e9074fa79fc6e2fe89ef69c: fix: should use newLLM to check delete timestamp (@bjwswang)
• 0e24420d2c3e9c8fe8172218433213a958c5459f: fix: skip invalid document in evaluation (@bjwswang)
• ba2310644408766e6f266689f4b0488806a6d387: fix: skip to create embedder if worker is utilizing vllm (@bjwswang)
• 8945fb5665e7a0e3ba19636fc21f22326d718a80: fix: the command arctl gen_test_dataset recursively reads files (@dayuy)
• 14b2b7649c5df5abdf97bf14eb8968d0bb20777b: fix: too long to call listModelServices (@0xff-dev)
• ab35f487e9faa3cdf95a48824648085a47fcecb4: fix: too many unexpected logs in arcadia-controller during startup (@0xff-dev)
• da0fb06ef4161e5342c7ab94251f5dfba3bb1725: fix: try to adapt unstrict csv as much as possible (@Abirdcfly)
• 4c851735a443c72aea5cc439402c31fcc1cf0808: fix: update qa.csv col (@Abirdcfly)
• e18063a542985162d13b628bb8010d595f31289f: fix: use simple(only qa) reference string as evaluation contexts (@bjwswang)
• 360066b20bc2fcd96aa64c6ee3a06c99d0feccfa: fix: when no related doc is found, should return retriever.spec.docNullReturn (@Abirdcfly)
• 9dae4b2e51c52f0a1891dfed60179c3434f8a71d: fix:The 'chunk_overlap' parameter did not take effect during text splitting. (@wangxinbiao)
• e2642ed3904779e7c13c93baae2e6e0f172c3da4: fix:Updating requirements (@)
• 60710c597a378b772f99e3ff7eb74641499a46be: fix:add an example on QA deduplicate (@)
• 811b208439e9482f37c160d892470884b4cf6280: fix:data processing mirror cannot be started. (@wangxinbiao)
• e275b5dd9ac3611aa16f5b3987d12c27c3a0c47f: fix:find the permissions needed for the command gen_test_dataset (@xxxxibo)
• 6fa120851dc10eaafc7e98d09054848858f7e6bc: fix:the field replicas display (@xxxxibo)
• 06dc31fa84320625b887e79f3528a81e020734c3: fix:typo (@xxxxibo)
• c5026591be3fff744bfcdec22add52787655f878: fix:update displayname in modelserver (@xxxxibo)

Others

• 97b05cdd67c3fcfd38c30a618b039314da416bf0: Add exportloopref linter (@Hoega)
• da5317d1a6101b098da6681b8c5f29e387942365: Fix exportloopref lint errors (@Hoega)
• f2d642a1d82482ef4fa3ade79ac573b5d6b9177f: Handle Incorrect Time (#540) (@y9rabbito)
• 7ff16855ac779e6249c4ae1ada112d797239e061: Merge remote-tracking branch 'upstream/main' (@nkwangleiGIT)
• fd2990492f4c74a66189d3b9244f5b5a458b8d0c: Update npm repo host and README (@nkwangleiGIT)
• a8345a245ece8949884cd8285aa51c4f5708a929: Use latest image for now and move agents to pkg (@nkwangleiGIT)
• bdf82d694c4e3dfe14bd20aa3a1855540548bc08: can't create pvc (@xxxxibo)
• 924a43f852738e604cd66c6fd62202461e0048f2: chore(appruntime): remove context key-value for app namespace (@bjwswang)
• 3389e17d781d739bcdf968ed05580ae487cfed00: chore: add evaluation CLI dockerfile (@Lanture1064)
• aebd054d48f58f0a3d36923f9e3b2d13e1924c88: chore: add limit to context len in eval data gen (@Lanture1064)
• 497f112658e573f03a3a56d4ef799b8a71ac4cbb: chore: add new models supported by zhipuai (@bjwswang)
• fa23aa7ea16318aa39738733424c2a07100a4ea3: chore: add omitpty to doc in ConversationDocsRespBody (@bjwswang)
• 0e3060708c5dee1cad1c9b6599775314dc5f494f: chore: add some logs in chat doc;increaset ingress timeout (@bjwswang)
• d6955fd2b76d78d7c4a2250fe70921552726bcfb: chore: add swagger doc (@xxxxibo)
• 0423dbb93ad32e35aa70af15108d1d0ea7f21992: chore: build eval image upon all pkg update (@Lanture1064)
• e81e7b17341fea6c3e5eff9db4e64f2101ba2757: chore: build eval image upon all pkg update (@Lanture1064)
• d2588eb0c6db1bdeebdeb47ae32ba2ab17570cdc: chore: fix cli.py (@Lanture1064)
• f669fe2864298102a16bd6e07148c6bf98306aad: chore: fix eval dockerfile & image ver align (@Lanture1064)
• cee947162e40b0015527264988ad08ac7c0b9619: chore: fix eval_image_build workflow (@Lanture1064)
• f26d48bd732eededebedc9c22c081971f20f30b6: chore: fix function call errors (@0xff-dev)
• ec1452936538c4c99db21d13f0b4dd23490be7b7: chore: fix ragas-once readme typo (@Lanture1064)
• f052495a7043d4d62a8d519488c80f271bf0784f: chore: rename data-processing to pypi (@bjwswang)
• 6568af007a7205dd834cd8cb961e9ef1eb4ce853: chore: revise ragas_once file structure (@Lanture1064)
• 6070d9bf28f71920516af3e36004cbeed155a6b7: chore: run pylint locally and fix lint issues (@bjwswang)
• c38505dc485f6141348a02ca68a97cbd3b7e8b73: chore: run pylint locally and fix lint issues (@wangxinbiao)
• 993e8f3a2a983344ef98667101edd94270a1397b: chore: set runAsUser to 0 for ray cluster sample and remove embedding for baichuan/qwen/chatglm (@bjwswang)
• fb4e00ceb8b8b72215dcc53ca86925bf8126d4e2: chore: update docs and use standard as the default storageclass (@bjwswang)
• bdc2192ec052d9ecb68875a5887c65eb60000d06: chore: update llms dockerfile (@Lanture1064)
• 64bafd9e9696a2b99dd0966b36dca4c72be6a44b: fix context in the prompt (@nkwangleiGIT)
• 2f779b27d838991fe2708226548de605a90119e9: refactor: change the report's scatter structure (@0xff-dev)
• 44025c702912528847e036c661154c3afd074c28: refactor: use same k8s client in apiserver and controller (@Abirdcfly)

New Contributors

• @y9rabbito made their first contribution in Handle Incorrect Time by y9rabbito · Pull Request #540 · kubeagi/arcadia
• @GHOSTHSP made their first contribution in feat:Real-time data access by GHOSTHSP · Pull Request #616 · kubeagi/arcadia
• @Hoega made their first contribution in Add Exportloopref linter and fix errors by Hoega · Pull Request #548 · kubeagi/arcadia

Thanks to our Contributors!

Thank you to everyone who contributed to v0.2.0! ❤️

And thank you very much to everyone else not listed here who contributed in other ways like filing issues, giving feedback, testing fixes, helping users in slack, etc. 🙏

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(Also a KubeBB Component) for KubeAGI Arcadia

A Helm chart(KubeBB Component) for KubeAGI Arcadia

A Helm chart(KubeBB Component) for KubeAGI Arcadia

A Helm chart(KubeBB Component) for KubeAGI Arcadia

A Helm chart(KubeBB Component) for KubeAGI Arcadia