ghrl | get url | first
https://github.com/iesahin/xvc/actions/runs/13875177382

🐢 We forgot to update the doc tests. Let’s run them again to update storage remove and file untrack commands.

🐇 There are issues with elision.

ghpl

╭───┬──────────────────────┬────────────────────┬──────────────────────────────╮
│ # │     headRefName      │       title        │             url              │
├───┼──────────────────────┼────────────────────┼──────────────────────────────┤
│ 0 │ storage-remove-16674 │ xvc storage remove │ https://github.com/iesahin/x │
│   │                      │                    │ vc/pull/270                  │
╰───┴──────────────────────┴────────────────────┴──────────────────────────────╯

🐢 Tests are passing; we can merge the PR. But our commit hook to check the CHANGELOG doesn’t work. That’s weird; we don’t get any errors when the CHANGELOG is not in the push set.

tmux new-window -c ($env.HOME | path join github.com iesahin xvc.py)  nvim 

🐢 Now we can update the Python bindings as well.

🐇 We forgot to bump package versions. We need to create a checklist for releases.

• ✅ #🌻 CREATE a release checklist (2025-03-25 17:52)

🐢 Let’s check if the latest version is updated on PyPI.

pypi xvc

🐇 Yes, it is.

$ ghrl | first
╭──────────────┬─────────────────────────────────────────────────────────╮
│ conclusion   │ failure                                                 │
│ displayTitle │ Add CLI completions                                     │
│ headBranch   │ clap-complete-16608                                     │
│ url          │ https://github.com/iesahin/xvc/actions/runs/13007688178 │
╰──────────────┴─────────────────────────────────────────────────────────╯

🐢 Although I turned off most of the watches, logs are still so large that it’s not possible to view them from the interface. Downloaded the log archive.

🦊 We can have different GitHub Actions steps for each test. Claude can help write such a repeating set of steps.

🐇 We can at least separate z_test_docs to see if integration tests or that fails.

🐢 We’ll have to add caching for test artifacts to upload them for coverage. I’m not sure we really need to add that complexity to the process just to avoid downloading the logs.

🐇 We can actually move all to Xvc. We need a GitHub Action to run an Xvc pipeline.

🐢 Eventually yes, we should move our testing to Xvc itself. Now, the logs show that there are differences in z_test_docs actually.

🐇 When I run the command below, it passes. We may have a different config in GitHub Actions.

$ XVC_TRYCMD_TESTS=storage,file,pipeline,core,start TRYCMD=overwrite rws cargo test --features test-ci -p xvc --test z_test_docs
test z_doc_tests ... ok
...
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 15.34s

🐢 We don’t have an XVC_TRYCMD_TESTS=storage,file,pipeline,core,start definition in GitHub Actions; let’s add it, bump the version, and try again.

$ cargo set-version "0.6.14-alpha.10"
   Upgrading xvc from 0.6.14-alpha.9 to 0.6.14-alpha.10
...

🦊 We could also update run-tests.zsh to get a quick response.

🐢 Yep, let’s do that as well.

🐇 Dev tests pass, but there are differences in the documents still. For some reason, the local run-tests.zsh doesn’t update xvc/book/src/ref/xvc-storage.md. It only has the help text as a reference, but it’s not updated with the aliases, and it breaks the CI. This is weird but a small issue. Fixed it manually.

$ ghrl | first 2
╭───┬────────────┬───────────────────┬────────────────────┬────────────────────╮
│ # │ conclusion │   displayTitle    │     headBranch     │        url         │
├───┼────────────┼───────────────────┼────────────────────┼────────────────────┤
│ 0 │            │ Add CLI           │ clap-complete-1660 │ https://github.com │
│   │            │ completions       │ 8                  │ /iesahin/xvc/actio │
│   │            │                   │                    │ ns/runs/1302770108 │
│   │            │                   │                    │ 2                  │
│ 1 │ success    │ Add CLI           │ clap-complete-1660 │ https://github.com │
│   │            │ completions       │ 8                  │ /iesahin/xvc/actio │
│   │            │                   │                    │ ns/runs/1302755462 │
│   │            │                   │                    │ 2                  │
╰───┴────────────┴───────────────────┴────────────────────┴────────────────────╯

🐢 The earlier one has passed. It looks like we’re ready to merge. Let’s update the CHANGELOG.md for release.

🐇 Setting the release version:

cargo set-version "0.6.14"
   Upgrading xvc from 0.6.14-alpha.10 to 0.6.14
...

🐢 Let’s check the CI

$ ghrl | first 2

╭───┬────────────┬───────────────────┬────────────────────┬────────────────────╮
│ # │ conclusion │   displayTitle    │     headBranch     │        url         │
├───┼────────────┼───────────────────┼────────────────────┼────────────────────┤
│ 0 │ success    │ Add CLI           │ clap-complete-1660 │ https://github.com │
│   │            │ completions       │ 8                  │ /iesahin/xvc/actio │
│   │            │                   │                    │ ns/runs/1302770108 │
│   │            │                   │                    │ 2                  │
│ 1 │ success    │ Add CLI           │ clap-complete-1660 │ https://github.com │
│   │            │ completions       │ 8                  │ /iesahin/xvc/actio │
│   │            │                   │                    │ ns/runs/1302755462 │
│   │            │                   │                    │ 2                  │
╰───┴────────────┴───────────────────┴────────────────────┴────────────────────╯

🐇 And merge:

$ ghpM --body $"(open CHANGELOG.md | lines | skip 2 | take 7)" --subject "Add completions" --squash

🐢 Tagged main and pushed. Packages should be built in a few minutes.

🐇 I think we can start by improving xvc.py. I mean, releasing. Yesterday we finished our work with the Rust library.

🐢 Then we can start to look at the GUI, I believe. We can replace that 3-column view with a table and preview. It will be much easier that way.

🐇 Yep. Let’s finish and release the Python version first. Then we’ll go on to the GUI.

🐢 Let’s take a look at the PRs first.

ghpl 
36	Bump pyo3 from 0.22.2 to 0.23.3	dependabot/cargo/pyo3-0.23.3	OPEN	2024-12-04T03:52:03Z

gh pr close 36
✓ Closed pull request iesahin/xvc.py#36 (Bump pyo3 from 0.22.2 to 0.23.3)

🐢 We have already upgraded to pyo3 0.23. No need for this. Let’s create a PR for the current branch.

git push --set-upstream origin v0.6.13

ghpC --fill
https://github.com/iesahin/xvc.py/pull/37

🐇 We can also tag and push the tags.

🐢 I’d like to have some more coverage for certain parts of the code. Let’s add some tests.

🐇 It looks like we mainly lack the storage tests. They need configuration to add keys to GitHub, and we already skip some of these even in the Rust code.

🐢 Umm, I see. We also need a way to measure coverage. Could we do this with Codecov, I wonder?

🐇 I found an example here: https://github.com/codecov/example-python/blob/main/.github/workflows/ci.yml. It needs coverage and pytest-cov in the requirements.

🐢 Let’s try this then.

🐇 Added coverage.yml file. It’s simpler than the other GitHub action. Need to update the token now.

🐢 There is an issue installing the requirements.

🐇 I forgot to add sudo to apt-get. Will take care of it now.

🐢 Let’s check the run.

ghrl
in_progress		Release v0.6.13	coverage	v0.6.13	pull_request	12556590550	2m31s	2024-12-31T06:59:58Z
in_progress		Release v0.6.13	publish-to-pypi	v0.6.13	pull_request	12556590548	2m31s	2024-12-31T06:59:58Z
...

🐇 It takes a while and we still didn’t add the CODECOV_TOKEN.

🐢 Let’s add it, and after that, we need to take a look at this blink configuration. It adds letters after the selection.

🐇 Added the secret and configured xvc.py for coverage. There is an error with the build, though. Maybe the command we should be using is maturin develop instead of build to make Xvc available for the environment.

🐢 Let’s update and try it then.

🐇 maturin develop requires a virtual environment.

🐢 I checked the options for build, and I think there is an option, but let’s search first.

🐇 I searched, but it looks like we can just pass an --out directory or install xvc from the target/wheels/ directory. The second option requires less maintenance.

🐢 Okay. Let’s add a step to the action then.

🐇 Now, let’s wait for the run to finish with gh run watch.

🐢 It failed. Let’s take a look at the logs:

ghrl
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12556939836	3m26s	2024-12-31T07:38:54Z

gh run view 12556939836 --log-failed

🐇 It looks like some of the tests are failing. Let’s run the tests locally.

🐢 We should run with the --forked option, and it looks like we have a test to update with the xvc file list.

🐇 Updated the test, and I noticed we forgot to supply the new --show-directories option in the Python interface.

🐢 Yep. Passing these options as command-line options in strings is not robust. It’s very easy to forget things. I think we should start using CLI structs directly, but it’s not time yet.

🐇 I agree. It’s one of the goals for building a GUI, actually.

🐢 The tests failed again.

ghrl | rg failure
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12557031091	3m49s	2024-12-31T07:51:19Z

gh run view 12557031091 --log-failed

🐇 There is a Git error now. We need to add a Git user and email to the action.

🐢 Added those and watching the results again now.

🐇 Why do you think the publish action always works? It can only run with the main branch, I think. No need to run it with other pushes.

🐢 Yes, let’s configure it now.

ghrl | rg failure
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12557083393	3m21s	2024-12-31T07:59:18Z

gh run view 12557083393 --log-failed

🐇 It looks like we also need xvc-test-helper in the path. Let’s cargo install it and add it to the path.

🐢 Added .cargo/bin to the path like:

- name: Add cargo bin to PATH
  run: echo "$HOME/.cargo/bin" >> $GITHUB_PATH

and installed the helper with cargo install xvc-test-helper.

🐇 By the way, GitHub Copilot is hallucinating about a method to update the path.

🐢 I searched and it may not be hallucinating. We can use the ::add-path:: command with echo, it looks like. This is new to me.

🐇 The tests failed again.

ghrl | rg failure
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12557208723	3m34s	2024-12-31T08:12:09Z

gh run view 12557208723 --log-failed

🐢 file().list() had a mistake, and we need to install rg for the tests.

🐇 Watching the test run. In the meantime, maybe we can review…

🐢 Failed again.

ghrl | rg failure
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12557294105	3m46s	2024-12-31T08:21:17Z

gh run view 12557294105 --log-failed

🐇 Added db.commit() to two places in the code. This should pass now.

🐢 Yeah!

ghrl | head -n 1
completed	success	Release v0.6.13	coverage	v0.6.13	pull_request	12557509836	3m46s	2024-12-31T08:44:33Z

gh run view 12557509836

✓ v0.6.13 coverage iesahin/xvc.py#37 · 12557509836
Triggered via pull_request about 4 minutes ago

JOBS
✓ linux in 3m36s (ID 35010253915)

ANNOTATIONS
! ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636
linux: .github#1


For more information about the job, try: gh run view --job=35010253915
View this run on GitHub: https://github.com/iesahin/xvc.py/actions/runs/12557509836

🐇 I can’t see a coverage report on codecov.io, though.

🐢 It says no coverage report is generated. Let’s test to generate XML files locally.

🐇 The option in the docs seems incorrect. --cov-branch doesn’t produce anything.

🐢 Let’s wait for the run again.

🐇 Should we add a badge to the README?

🐢 It won’t show much, but yeah, let’s make it.

🐇 The results are in and it shows 100% coverage. This means it doesn’t actually test anything.

🐢 We need Rust coverage for this. Let’s search for it.

🐇 I found this: https://github.com/cjermain/rust-python-coverage. It runs cargo llvm-cov with the project and measures test coverage. But we don’t have any Rust tests.

🐢 It looks like we don’t need Rust tests. cargo llvm-cov can check coverage with the Python as well.

$ cargo llvm-cov show-env --export-prefix
export RUSTFLAGS=" -C instrument-coverage --cfg coverage --cfg trybuild_no_target"
export LLVM_PROFILE_FILE="/home/.../rust-python-coverage/target/rust-python-coverage-%m.profraw"
export CARGO_INCREMENTAL="0"
export CARGO_LLVM_COV_TARGET_DIR="/home/.../rust-python-coverage/target"

🐢 Let’s take a look at what remained for 0.6.13.

🐇 I think there is nothing left. Python must be published when we merged the PR.

🐢 Let’s take a look by searching xvc python.

🐇 The PyPI page is https://pypi.org/project/xvc/ and it still reports the version as 0.6.11. There must be something.

🐢 Now let’s take a look at

https://github.com/iesahin/xvc.py

🐇 The run seems to be OK though.

https://github.com/iesahin/xvc.py/actions/runs/12557780141/job/35010938462

🐢 Maybe the version in pyproject.toml is still 0.6.11 and we forgot to update it?

tmux new-window -c $HOME/github.com/iesahin/xvc.py/ nvim

🐇 There is no version string in ~/github.com/iesahin/xvc.py/pyproject.toml. It’s dynamic.

🐢 Then, let’s try to publish from local now.

🐇 The iex username requires email verification.

🐢 It looks like I publish xvc through the iesahin account, not iex. Maybe I can add both accounts to the project.

🐇 There seem to be no errors on the PyPI site.

🐢 Updating the token. Let’s add the token to pass to run maturin publish.

🐇 We’re receiving invalid or non-existent authentication information. Upgraded maturin to see if it fixes the issue.

🐢 We can also double-check the key.

🐇 Uploaded successfully from local. Let’s check the job again.

🐢 The release job was skipped because we didn’t tag after the merge. https://github.com/iesahin/xvc.py/actions/runs/12557780141/job/35011303045 That looks like the reason.

🐇 I should be more careful which is run and which is skipped.

🐢 Maybe we can relax the condition. We do this rarely. Maybe republishing is alright?

🐇 Yep, removed that. The jobs are running now. Let’s watch them to see what happens when we publish some of the packages.

🐢 In the meantime, let’s experiment with searching commands file with fzf-lua.

🐇 It requires more experimentation, but we can start from https://github.com/ibhagwan/fzf-lua/wiki/Advanced#interactive-shell-command.

🐢 The xvc publish jobs failed, btw.

ghrl
completed	failure	Remove if condition from release	publish-to-pypi	v0.6.13	push	12568863716	15m26s	2025-01-01T08:30:03Z

gh run view 12568863716 --log-failed
...
Release	Run actions/download-artifact@v4.1.7	        Please ensure that your artifact is not expired and the artifact was uploaded using a compatible version of toolkit/upload-artifact.
...

🐇 It was using an older version of upload-artifact.

🐢 Rerunning the job and it looks like it runs for both push to main and tag with v0.6.13. We can turn off push to main, I believe.

🐇 There was a missing upload-artifact again. Fixed and updated the tags.

ghrl completed failure update upload-artifacts publish-to-pypi main push 12569060915 14m39s 2025-01-01T08:57:22Z

ghrf 12569060915

🐢 There are conflicts with uploaded artifacts now. We may need to clean up the artifacts manually.

🐇 The conflicts were not about inter-workflow names. The names were conflicting because all files were named wheel. I added the platform and target to the names to avoid conflicts.

🐢 Let’s wait then. Maybe it will work this time. Could you search for a Lua console for Neovim?

🐇 Let’s try this one: return { “yarospace/lua-console.nvim”, lazy = true, keys = “`”, opts = {}, }

🐢 I couldn’t make it run but won’t spend much time ATM. How about the jobs?

ghrl
completed	failure	update upload artifact names	publish-to-pypi	v0.6.13	push	12569298414	13m57s	2025-01-01T09:27:00Z

ghrf 12569298414
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9180510Z ##[group]Run actions/download-artifact@v4
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9182098Z with:
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9182859Z   name: wheels
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9183843Z   merge-multiple: false
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9184817Z   repository: iesahin/xvc.py
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9185820Z   run-id: 12569298414
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9186903Z ##[endgroup]
Release	Run actions/download-artifact@v4	2025-01-01T09:40:53.1937617Z Downloading single artifact
Release	Run actions/download-artifact@v4	2025-01-01T09:40:53.4694769Z ##[error]Unable to download artifact(s): Artifact not found for name: wheels
Release	Run actions/download-artifact@v4	        Please ensure that your artifact is not expired and the artifact was uploaded using a compatible version of toolkit/upload-artifact.
Release	Run actions/download-artifact@v4	        For more information, visit the GitHub Artifacts FAQ: https://github.com/actions/toolkit/blob/main/packages/artifact/docs/faq.md

🐇 Now the download doesn’t work.

🐢 Update it to download using patterns.

🐇 Did so and let’s take a look at the jobs again.

ghrl
completed	failure	added pattern to download	.github/workflows/publish.yml	main	push	12569456859	0s	2025-01-01T09:49:19Z

ghrv 12569456859

X main .github/workflows/publish.yml · 12569456859
Triggered via push about 3 minutes ago

X This run likely failed because of a workflow file issue.

For more information, see: https://github.com/iesahin/xvc.py/actions/runs/12569456859

🐢 The line with the pattern was reported as broken.

🐇 Removed it and recommitted.

ghrl
✅ completed	success	remove pattern to download all	publish-to-pypi	v0.6.13	push	12569509435	14m41s	2025-01-01T09:58:49Z

🐢 And now this completes the v0.6.13 release.

🐇 We’ll see how it will work next time.

🐢 Yep. Let’s move on to the GUI for now. Is that okay with you?

🦊 There is a brew create command, but could we really copy it from someone else?

🐢 I read the espanso example, and I’ll start by adding a repository. Welcome https://github.com/iesahin/homebrew-xvc

🦊 We can search to automate this for Rust stuff first. Maybe it’s easier to automate than to do it manually with examples.

🐢 Let’s search “how to automate homebrew tap formula updates with github actions”

🐲 There is a shell script that updates the brew repo with a shell script. It fires the action from the main repository with a command like:

gh workflow run release.yml -f version=${{ env.VERSION }} -R itspriddle/homebrew-slack-notify

🐢 It looks a little brittle, though.

🐲 There is also a GitHub action: https://github.com/marketplace/actions/homebrew-tap but it doesn’t look very popular. There is another one https://github.com/marketplace/actions/bump-homebrew-formula but this is for formulas, or the default settings are those.

🐢 There is an example https://github.com/marketplace/actions/bump-homebrew-formula#examples for taps as well.

🐲 This one is simpler, and probably we can just add this first: https://github.com/marketplace/actions/homebrew-bump-formula Most of the fields are optional.

🐇 I think this is much easier than the other. Let’s start with this.

🐢 Now let’s add a token to the Xvc repo. ✅

🦊 Add a branch to Xvc and add the release action file.

🐢 Added the release action file ~/github.com/iesahin/xvc/.github/workflows/homebrew.yml, but we still don’t have a tap; maybe we can just add one.

🦊 Can you check espanso’s for example?

🐢 It has a cask and is in the core. We need a tap example.

🐲 I believe we need to clone homebrew-xvc and run gh workflow run.

🐢 Yes, let’s clone the repo and try to run it manually.

🐇 We don’t have a formula yet.

🦊 What’s the directory structure of a Homebrew tap project?

🐢 It’s something like this, but we don’t need tests for this, I believe.

.
├── Formula
│   └── <formula_name>.rb
├── LICENSE
├── README.md
└── .github
    └── workflows
        └── tests.yml

🐇 What goes into the Formula?

🦊 Let’s take a look at ripgrep’s example:

class Ripgrep < Formula
  desc "Search tool like grep and The Silver Searcher"
  homepage "https://github.com/BurntSushi/ripgrep"
  url "https://github.com/BurntSushi/ripgrep/archive/refs/tags/14.1.1.tar.gz"
  sha256 "4dad02a2f9c8c3c8d89434e47337aa654cb0e2aa50e806589132f186bf5c2b66"
  license "Unlicense"
  head "https://github.com/BurntSushi/ripgrep.git", branch: "master"

  livecheck do
    url :stable
    strategy :github_latest
  end

  bottle do
    sha256 cellar: :any,                 arm64_sequoia:  "b8bf5e73c9c9b441de067ec86ac167b071ecc2078dcb1d89d2cebbb151feab35"
    sha256 cellar: :any,                 arm64_sonoma:   "47b9c3515c866b147f0e98735cab165d6471b9f28fab1ba2c57e59c43da5c10b"
    sha256 cellar: :any,                 arm64_ventura:  "e14a94e84c028ff53c1be3b106fdeb5aca4d7c893a819e7fb967e0719b946a28"
    sha256 cellar: :any,                 arm64_monterey: "ad8dc4ab475c84e2a1e60f5b3107f52dd59e33f84a08284b19681d8b98508fd7"
    sha256 cellar: :any,                 sonoma:         "71d434eeabc2af220285b037f7264563ce9bc77a41af35eabe2213276a37ec2b"
    sha256 cellar: :any,                 ventura:        "0cdb547c696992d08c6613c40934218964f4a061b5413c4b2f013c3f0c3ed253"
    sha256 cellar: :any,                 monterey:       "2ce54302e4524ad28389aca5a16333d4193128e911de2881e6b0e953559d89cd"
    sha256 cellar: :any_skip_relocation, x86_64_linux:   "97d7cbd33b4d0ed09551e3dbc07f830d3df018c2aefbb2222a12ccfb829aae30"
  end

  depends_on "asciidoctor" => :build
  depends_on "pkgconf" => :build
  depends_on "rust" => :build
  depends_on "pcre2"

  def install
    system "cargo", "install", "--features", "pcre2", *std_cargo_args

    generate_completions_from_executable(bin/"rg", "--generate", shell_parameter_format: "complete-")
    (man1/"rg.1").write Utils.safe_popen_read(bin/"rg", "--generate", "man")
  end

  test do
    (testpath/"Hello.txt").write("Hello World!")
    system bin/"rg", "Hello World!", testpath
  end
end

🐇 What’s in that tar file?

wget https://github.com/BurntSushi/ripgrep/archive/refs/tags/14.1.1.tar.gz 
--2025-01-03 06:28:07--  https://github.com/BurntSushi/ripgrep/archive/refs/tags/14.1.1.tar.gz
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/BurntSushi/ripgrep/tar.gz/refs/tags/14.1.1 [following]
--2025-01-03 06:28:08--  https://codeload.github.com/BurntSushi/ripgrep/tar.gz/refs/tags/14.1.1
Resolving codeload.github.com (codeload.github.com)... 140.82.121.10
Connecting to codeload.github.com (codeload.github.com)|140.82.121.10|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘14.1.1.tar.gz’

     0K .......... .......... .......... .......... ..........  355K
    50K .......... .......... .......... .......... .......... 1.05M
   100K .......... .......... .......... .......... .......... 1.35M
   150K .......... .......... .......... .......... .......... 3.61M
   200K .......... .......... .......... .......... ..........  876K
   250K .......... .......... .......... .......... .......... 6.50M
   300K .......... .......... .......... .......... .......... 1.15M
   350K .......... .......... .......... .......... .......... 1.65M
   400K .......... .......... .......... .......... .......... 4.88M
   450K .......... .......... .......... .......... .......... 1.22M
   500K .......... .......... .......... .......... .......... 1.56M
   550K .......... .......                                     3.62M=0.5s

2025-01-03 06:28:09 (1.21 MB/s) - ‘14.1.1.tar.gz’ saved [581402]

mv 14.1.1.tar.gz $HOME/Downloads/ripgrep-14.1.1.tar.gz

tar xvzf $HOME/Downloads/ripgrep-14.1.1.tar.gz 

🐢 It’s a source distribution. It doesn’t contain any binaries.

🦊 We can use the same URL format, it looks. Let’s try this:

wget https://github.com/iesahin/xvc/archive/refs/tags/0.6.13.tar.gz 
--2025-01-03 06:34:00--  https://github.com/iesahin/xvc/archive/refs/tags/0.6.13.tar.gz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/iesahin/xvc/tar.gz/refs/tags/0.6.13 [following]
--2025-01-03 06:34:01--  https://codeload.github.com/iesahin/xvc/tar.gz/refs/tags/0.6.13
Resolving codeload.github.com (codeload.github.com)... 140.82.121.9
Connecting to codeload.github.com (codeload.github.com)|140.82.121.9|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2025-01-03 06:34:01 ERROR 404: Not Found.

🐢 Our source should be in another location.

🐇 We had an unpublished release. Didn’t we release the binaries a few days ago?

🐢 Umm, there must be something about the gh command where we set draft=false, but seemingly it didn’t work out.

🐇 Ok, the URL is something like:

wget https://github.com/iesahin/xvc/archive/refs/tags/v0.6.13.tar.gz 

🐢 Ok, this works. We’ll use this one for the URL.

🐇 What should we put for SHA256?

🦊 Let’s check ripgrep’s again.

🐢 Downloading and using xvc file hash (alias xvcfh):

wget https://github.com/BurntSushi/ripgrep/archive/refs/tags/14.1.1.tar.gz 
...

xvc file hash -a sha2 14.1.1.tar.gz
4dad02a2f9c8c3c8d89434e47337aa654cb0e2aa50e806589132f186bf5c2b66	14.1.1.tar.gz

🐢 Yep, this matches the source.

🦊 Then we can just use the same.

🐇 I want to learn how to download to $TMPDIR with wget.

🐢 The -P option is used for this.

wget -P $TMPDIR https://github.com/iesahin/xvc/archive/refs/tags/v0.6.13.tar.gz 
--2025-01-03 06:55:02--  https://github.com/iesahin/xvc/archive/refs/tags/v0.6.13.tar.gz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/iesahin/xvc/tar.gz/refs/tags/v0.6.13 [following]
--2025-01-03 06:55:03--  https://codeload.github.com/iesahin/xvc/tar.gz/refs/tags/v0.6.13
Resolving codeload.github.com (codeload.github.com)... 140.82.121.9
Connecting to codeload.github.com (codeload.github.com)|140.82.121.9|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘/var/folders/gf/18gghtz15_zf0cr8v6dymglw0000gn/T/v0.6.13.tar.gz’
...
2025-01-03 06:55:07 (2.59 MB/s) - ‘/var/folders/gf/18gghtz15_zf0cr8v6dymglw0000gn/T/v0.6.13.tar.gz’ saved [8785449]

xvcfh -a sha2 $TMPDIR/v0.6.13.tar.gz
01bee5d840eefec7be1f52cc75546e1ffd7e332dfac83d875655f84e61e3a9f6	/var/folders/gf/18gghtz15_zf0cr8v6dymglw0000gn/T//v0.6.13.tar.gz

🐇 According to documentation, bottles are produced by Brew itself, and the documentation around bottling taps is limited.

🦊 We can just start with the source distribution. We can test it now.

🐢 The source distribution seems to work, but it requires Rust, and Brew downloads everything related to it.

🐇 I think that may be enough for the time being. With 0.6.14, we can work on providing bottles.

🐢 Umm, yes. I’m already bored with this stuff.

🐇 It looks like you first need to turn off this ghost text from the blink output. It makes writing insufferable.

🐢 Yep, let’s do that first.

🐇 Now, let’s restart Neovim.

🐢 I don’t know why blink.cmp doesn’t prioritize the emojis I use. Maybe we can just use #tor and #rab for ourselves.

🐇 I think over time it will learn that the emojis I defined in the snippets file should have higher priority, but let’s skip this for now. What do we need to do to add cross-compilation?

🐢 Maybe we can just make the completion menu wait a bit longer. It shows up almost instantly, and I want it to wait for a few more milliseconds.

🐇 Okay, let’s look at the config.

🐢 The configuration file doesn’t seem to have a key for this. Let’s search: blink.cmp.

🐇 I think the culprit is typo resistance; it causes better options to be pushed down: https://cmp.saghen.dev/configuration/reference#fuzzy

🐢 Let’s turn that off.

🐇 There is an error in the configuration file. I don’t know why it’s failing.

🐢 I added emojis to Espanso and checked the error message. The configuration I copied from their docs seems to be broken. The message is:

...share/nvim/lazy/blink.cmp/lua/blink/cmp/config/utils.lua:14: fuzzy.max_items: unexpected field found in configuration

I’ll just delete that line.

🐇 It still doesn’t prioritize snippets, but we can look into this later. Espanso seems to be a better tool for this anyway.

🐢 Yep. Let’s look into cross-compilation support for Rust.

🐇 The well-known option is cross.rs: https://github.com/cross-rs/cross

🐢 We can start with that. Installation is from the Git repository:

$ cargo install cross --git https://github.com/cross-rs/cross
    Updating git repository `https://github.com/cross-rs/cross`
    Updating git submodule `https://github.com/cross-rs/cross-toolchains.git`
  Installing cross v0.2.5 (https://github.com/cross-rs/cross#4090beca)
...
   Installed package `cross v0.2.5 (https://github.com/cross-rs/cross#4090beca)` (executables `cross`, `cross-util`)

🐇 Now we have the cross and cross-util commands. Cross-compilation requires Podman on Linux or Docker on macOS. Do we have Docker?

🐢 It looks like we don’t. Maybe we can just set up a remote build using Podman or GitHub Actions. I saw a crate for that yesterday; I remember saving it somewhere but can’t find it now. Searching again seems easier, which says something about my archival and retrieval habits.

🐇 Maybe later you can add some vector search capabilities to your archive—semantic search.

🐢 Yep, sometime later.

🐇 Now let’s search for “adding rust cross compilation to github actions.”

🐢 I found a link to the action: https://github.com/marketplace/actions/build-rust-projects-with-cross

Let’s look at the example:

jobs:
  release:
    name: Release - ${{ matrix.platform.os-name }}
    strategy:
      matrix:
        platform:
          - os-name: FreeBSD-x86_64
            runs-on: ubuntu-20.04
            target: x86_64-unknown-freebsd
            skip_tests: true

          - os-name: Linux-x86_64
            runs-on: ubuntu-20.04
            target: x86_64-unknown-linux-musl

          - os-name: Linux-aarch64
            runs-on: ubuntu-20.04
            target: aarch64-unknown-linux-musl

          - os-name: Linux-riscv64
            runs-on: ubuntu-20.04
            target: riscv64gc-unknown-linux-gnu

          - os-name: Windows-x86_64
            runs-on: windows-latest
            target: x86_64-pc-windows-msvc

          - os-name: macOS-x86_64
            runs-on: macOS-latest
            target: x86_64-apple-darwin

          # more targets here ...

    runs-on: ${{ matrix.platform.runs-on }}
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Build binary
        uses: houseabsolute/actions-rust-cross@v0
        with:
           command: ${{ matrix.platform.command }}
          target: ${{ matrix.platform.target }}
          args: "--locked --release"
          strip: true
      - name: Publish artifacts and release
        uses: houseabsolute/actions-rust-release@v0
        with:
          executable-name: ubi
          target: ${{ matrix.platform.target }}

🐇 We can just convert the current configuration to this and see if it works.

🐢 Let’s do that. I created .github/workflows/release.yml and will update it.

🐇 Let’s check the results with gh:

$ gh -R iesahin/xvc run list
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525070531	34s	2024-12-28T08:17:28Z

🐢 It says the release has completed.

$ gh -R iesahin/xvc run view 12525070531

X v0.6.13 Release iesahin/xvc#263 · 12525070531
Triggered via pull_request about 3 minutes ago

JOBS
X Release - FreeBSD-x86_64 in 12s (ID 34936257653)
  ✓ Set up job
  ✓ Checkout
  X Build binary
  - Publish artifacts and release
  ✓ Post Build binary
  ✓ Post Checkout
  ✓ Complete job
...

Now let’s look at the failure:

$ gh -R iesahin/xvc run view 12525070531 --log-failed

🐇 Change the order and see what happens. Maybe we should first succeed in a non-cross-compilation build.

$ gh -R iesahin/xvc run list
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525178664	35s	2024-12-28T08:34:18Z
...
$ gh -R iesahin/xvc run view   12525178664 --log-failed
...

🐢 It’s the same error. The usage example is actually much more sophisticated.

🐇 The issue is that we’re asking for command from the matrix, but the matrix doesn’t define it. That’s the second time today an example from the documentation has failed. I’ve added command for each platform. We can also add separate features this way to make platform-specific functionality work.

🐢 Agreed. Let’s look at it once more.

$ gh -R iesahin/xvc run list | rg Release | head -n 1
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525253982	41s	2024-12-28T08:46:23Z

$ gh -R iesahin/xvc run view 12525253982 --log-failed
...
Release - macOS-x86_64	Build binary	2024-12-28T08:46:44.8464010Z  [1m [31merror [0m [1m: [0m the lock file /Users/runner/work/xvc/xvc/Cargo.lock needs to be updated but --locked was passed to prevent this

🐇 Ah, that’s a different error. Let’s remove the --locked flag and retry.

$ gh -R iesahin/xvc run list | rg Release | head -n 1
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525289176	1m8s	2024-12-28T08:53:23Z

🐢 We’re finally starting to get some good news. Now we’re getting OpenSSL errors. We need feature flags for these platforms or to specify where OpenSSL is. Let’s add bundled-openssl to the failed ones. We also need bundled-sqlite for Windows binaries.

gh -R iesahin/xvc run list | rg Release | head -n 1
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525354179	3m9s	2024-12-28T09:04:55Z

🐇 It looks like the Changes.md file is missing, and it can’t upload the binaries as a release because of this. I’ll set the changes file to CHANGELOG.md.

🐢 It’s weird to fail because of that. I think we should report these errors and possibly send a PR to make that file optional.

gh -R iesahin/xvc run list | rg Release | head -n 1
in_progress		v0.6.13	Release	v0.6.13	pull_request	12525449368	1m38s	2024-12-28T09:18:40Z
...
✓ Release - macOS-x86_64 in 2m39s (ID 34937017695)
✓ Release - macOS-aarch64 in 2m42s (ID 34937017787)
..
ARTIFACTS
xvc-macOS-x86_64.tar.gz
xvc-macOS-arm64.tar.gz

🐢 It looks like Linux-riscv64 has OpenSSL compilation errors. I think we can skip this platform for now. The goal was to add aarch64 for macOS, and that seems to have succeeded.

gh -R iesahin/xvc run list | rg Release | head -n 1
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525504079	3m24s	2024-12-28T09:26:43Z
...
X Release - Linux-x86_64	Build binary	2024-12-28T09:29:16.2262518Z  [0m [1m [38;5;9merror[E0308] [0m [0m [1m: mismatched types [0m
...

🐢 It looks like Linux-x86_64 doesn’t support reflinks. Maybe we can make reflinks an optional feature and add it specifically to macOS and Windows targets.

🐇 I’ve removed reflink from the default features. This is a breaking change, but fortunately we don’t have many users who will be affected by it.

Let’s check the results once more.

gh -R iesahin/xvc run list | rg Release | head -n 1
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525597866	3m39s	2024-12-28T09:42:44Z

🐢 Now turn off the NetBSD target as well.

🐇 FreeBSD and Linux-x86_64 targets are building. Let’s see why the ARM Linux targets are failing.

gh -R iesahin/xvc run list | rg Release | head -n 1
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525643284	4m46s	2024-12-28T09:52:04Z

🐇 It looks like those targets don’t have libsqlite3 installed. Let’s add bundled-sqlite to these targets too.

gh -R iesahin/xvc run list | rg Release | head -n 1
completed	failure	v0.6.13	Release	v0.6.13	pull_request	12525734325	4m11s	2024-12-28T10:07:36Z

🐢 Adding Android as a target didn’t work. Let’s remove it for now; it seems to require more work.

🐇 I think we’ll eventually move to using Xvc to distribute binaries. We could build the Android binary on Termux and link it on the releases page or push it as a release artifact.

🐢 I need to learn more about GitHub releases. If we can add artifacts to the release, maybe we can do some of this work locally.

🐇 We can start by looking at the capabilities of gh commands.

🐢 It looks like it’s possible to upload assets to releases. Let’s check how that works.

gh release upload --help

🐇 So basically, we can just upload files to tags. We can list releases and work with them like anything else.

gh -R iesahin/xvc release list

And we can delete releases:

for r in v0.4.2-alpha.8 v0.4.2-alpha.7 v0.4.2-alpha.6 v0.4.2-alpha.5 v0.4.2-alpha.0 v0.4.1-alpha.0; do
  gh -R iesahin/xvc release delete "${r}"
done

🐢 Now let’s list them again.

gh -R iesahin/xvc release list

🐇 The latest one has failed again.

X Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

🐢 It looks like we’re coming to the end of this session. Under what conditions will we create a release?

🐇 I think it’s better to release only non-alpha tags.

🐢 Then the rule in the workflow will be something like:

on:
  workflow_dispatch:
  push:
    tags:
      - "v*.*.*"
      - "!v.*.*-alpha.*"

🐇 And now we have this result:

✓ v0.6.13 Release iesahin/xvc#263 · 12533498361
Triggered via pull_request about 21 minutes ago

JOBS
✓ Release - FreeBSD-x86_64 in 4m14s
✓ Release - Linux-x86_64 in 4m3s
✓ Release - Linux-aarch64 in 4m50s
✓ Release - Windows-x86_64 in 8m47s
✓ Release - Windows-aarch64 in 6m42s
✓ Release - macOS-x86_64 in 2m37s
✓ Release - macOS-aarch64 in 2m36s

🐢 Nice! Let’s check the release list.

gh -R iesahin/xvc release list

🐇 Why is there no “latest” release?

🐢 We have to tag it first.

🐇 Ah, right. Let’s tag it then.

🐢 I’ve pushed the changes and tagged them with v0.6.13-alpha.5.

🐇 I think it’s possible to make a release today.

🐢 It looks like it, yes.

🐇 We can merge the PR and tag it. Then everything should work.

🐢 We need some cleanup in the YAML files, though.

🐇 We can leave that for the next release.

gh -R iesahin/xvc run list
in_progress		v0.6.13	Rust-CI	v0.6.13	pull_request	12533690474	9m18s	2024-12-29T08:02:29Z
completed	success	Release	Release	v0.6.13-alpha.5	push	12533689099	9m11s	2024-12-29T08:02:19Z

🐢 Now we have another failure in the regular CI. Let’s look at it.

🐇 The issue seems to be in the doc tests:

- Total #: 8 Workspace Size:         276 Cached Size:          19
+ Total #: 8 Workspace Size:         278 Cached Size:          19

🐢 These tests are brittle, but they provide valuable information. Let’s fix it and push again.

🐇 Done. We can also remove some of the watches that produce so many logs.

🐢 I’m a bit ambivalent about them. I thought we could use these watches when debugging, but experience has shown that we need more granular watches during debugging and almost never use these otherwise. Let’s remove some of them.

🐇 We can increase the output for certain commands, but the tracing output doesn’t help much in regular runs. If Xvc gets popular enough that we can’t cope with bug reports, we can always add more watches.

🐢 Another option is to exclude the watch code from the release build, but that won’t change anything for our debug cycles.

🐇 I think removing them is a fair trial. We can always put them back when debugging.

🐢 Watches could also produce regular output instead of tracing. That way we won’t forget to remove them.

🐇 Ah, yep, that’s a good option too.

🐢 We can have a trace! macro similar to the current one for user consumption, and a watch! macro that sends output to stderr.

🐇 Good idea. Let’s do that in the next release.

🐢 Let’s check the tests before pushing this cleanup.

gh -R iesahin/xvc run list
completed	success	v0.6.13	Rust-CI	v0.6.13	pull_request	12533993607	13m49s	2024-12-29T08:47:50Z

🐇 CI succeeded, and the release didn’t run. Let’s do some more cleanup.

gh -R iesahin/xvc run list
in_progress		v0.6.13	Rust-CI	v0.6.13	pull_request	12534416152	43s	2024-12-29T09:57:02Z

Can't locate IPC/Cmd.pm in @INC (@INC contains: /home/runner/work/xvc.py/xvc.py/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-844a96d66ae533b1/out/openssl-build/build/src/util/perl /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 . /home/runner/work/xvc.py/xvc.py/target/x86_64-unknown-linux-gnu/release/build/openssl-sys-844a96d66ae533b1/out/openssl-build/build/src/external/perl/Text-Template-1.56/lib)

The issue seemed to be a missing Perl package. Building rust-openssl now appears to require the perl-core package.

I added apt-get update && apt-get install perl-core to the CI configuration, but that didn’t work.

However, as this GitHub issue comment suggests, Maturin with its manylinux support uses different kinds of Docker containers to build the packages. We must detect whether it is a CentOS or Debian-based container and install the missing packages accordingly.

The Building wheels step in the configuration should look similar to:

- name: Build wheels
  uses: PyO3/maturin-action@v1
  with:
    target: ${{ matrix.target }}
    manylinux: auto
    args: --release --out dist
    before-script-linux: |
      # If we're running on RHEL/CentOS, install needed packages.
      if command -v yum &> /dev/null; then
          yum update -y && yum install -y perl-core openssl openssl-devel pkgconfig libatomic

          # If we're running on i686, we need to symlink libatomic
          # in order to build openssl with the -latomic flag.
          if [[ ! -d "/usr/lib64" ]]; then
              ln -s /usr/lib/libatomic.so.1 /usr/lib/libatomic.so
          fi
      else
          # If we're running on a Debian-based system.
          apt update -y && apt-get install -y libssl-dev openssl pkg-config
      fi

This will run the specified script before executing the Maturin build command in the container, ensuring all missing packages are installed.

🐢 The tests for yesterday’s Git integration PR failed. I’ll begin by checking the logs.

🐇 Maybe run the tests locally to see. They might fail on your machine as well. It might be a simple thing.

🐢 Probably, yes. I’ve started the tests now. I shouldn’t forget to run them before testing the PR; it may save some time.

🐇 Ah, yeah, maybe. Yesterday you were already at the end of the workday, so it didn’t matter much. But you can shorten the testing time by reducing the number of files, etc. I think a separate benchmark suite might be good to have. Tests should be shorter; benchmarks should only run for tags.

🐢 Yup. I should shorten the tests. I should make them use a shorter list of files, maybe.

🐇 Local tests have passed. You’ll have to check the logs.

🐢 It seems git diff --name-only --cached returns files not yet added. That’s causing an error in new repositories with stash. Stashing shouldn’t run at all when there are no staged files.

🐇 Git behavior between the local version and the remote version seems different.

🐢 I think the trace should also show the Git version string.

🐇 You should also require a minimum Git version for the commands. You’re using some newer options, and not all users may have them.

🐢 Yep. Let’s see the version on the CI now.

🐢 Ah, I see—the problem is not about versions. It’s that the CI doesn’t configure git config --global user.email and user.name. Commits don’t work without these.

🐇 You should remove the Git version report, then. It’s one more process call for no reason.

🐢 I think it should be in trace!; I can check the verbosity level and call it only if it’s trace.

🐢 I completed the xvc-config docs as well. I think I can merge it before the tests finish, as it’s just a documentation update.

🐇 You seem to be rushing for the release.

🐢 Yeah, I want to really start testing this on the servers.

🐇 Then go ahead; let’s release a new version.

🐇 It looks like you’re back for an evening session. I think it’s time to start using Xvc on your torrent server.

🐢 Ah, yeah. Let’s see how it goes.

🐇 Create a repository for torrents. Then you can add files to it with cache-type = symlink or cache-type = hardlink.

🐢 I think creating a local storage may also help. I can use it to store files to retrieve them later.

🐇 Umm. We don’t have a “garbage collection” facility yet, you know. There are no file deletions at the moment. It may be better to have a repository-to-repository transfer feature, using SSH.

🐢 Yeah, we don’t have SSH storage either. I think this highlights a lack of features.

🐇 It’s possible to mimic Rsync storage with xvc storage new generic, but it doesn’t feel quite the same.

🐢 Then I’m adding these three tickets.

🐇 I think for 0.3.4, the command we add might be xvc file delete. You can work on this and add rsync support as well.

🐢 There is a librsync bindings library for Rust. But it looks like it doesn’t allow transferring file contents between hosts. There is also rusync, which is similar to rsync and implemented in Rust. There is also fast_rsync in pure Rust. It uses MD4 to calculate deltas, though, I think.

🐇 None of these seem to have network capability, though.

🐢 I think it’s better just to use the process for now. We are trying to come up with the simplest solution for now. We are just trying to be more general.

🐇 Yes, I think we can just use the process for the time being.

🐢 Then let’s start by adding it.

🐇 We should also begin to create release notes. GitHub can generate them from PRs. Automatically generated release notes

🐢 Created an issue for that. Now we’re starting xvc storage new rsync #111.

🐇 Ok. Let’s do this.

🐢 I’ve merged PR#66. Let’s list the outstanding issues now.

65	OPEN	Add Mermaid Support to Netlify Installation
64	OPEN	Fix `xvc-storage` compilation warnings	bug
54	OPEN	Fix all references to `xvc data` in the documentation
51	OPEN	Add a benchmark script to compare Xvc with other tools
50	OPEN	Consider changing `xvc file push` to `xvc file send` and `xvc file pull` to `xvc file retrieve`
49	OPEN	Add `--generic-command` as a dependency type to `xvc pipeline dependency`
48	OPEN	Write rsync example for `xvc storage generic`	documentation
47	OPEN	Write rclone example for `xvc storage generic`	documentation
46	OPEN	Allow to skip init in remotes with `--skip-init` option
45	OPEN	Update `VStore::to_store` to use some internal mechanism to avoid `XvcStore::insert`
43	OPEN	Update to `clap 4.0`
42	OPEN	Clean up xvc crate dependencies
36	OPEN	Add version to released file binaries
33	OPEN	Update `arch/remotes.md` for new naming	documentation
32	OPEN	Remove arc42 sections from the documentation	documentation
29	OPEN	Add documentation for `xvc storage new gcs`	documentation
28	OPEN	`xvc storage new yandex`	enhancement
24	OPEN	Create a logo for Xvc	documentation
23	OPEN	Add a Github action to upload new versions to crates.io	automation
20	OPEN	ref: Add rsync example for `xvc storage new generic`
18	OPEN	Add a new workflow for remote tests	automation
12	OPEN	Create a website for Xvc
5	OPEN	Add storage tests to Github Actions	automation
4	OPEN	fix clippy warnings	bug
1	OPEN	`xvc storage new` for all S3 compatible cloud services supported by `rust-s3`

🐇 As far as I know, you removed the arc42 stubs from the docs yesterday. You can close issue #32.

🐢 I closed it via gh issue close.

🐇 You should add a comment to that.

🐢 Added the comment.

🐇 It’s October 30th. Looking at the logs, you’ve closed some more issues since we last talked.

🐢 I’ve been productive over the last few days. Now, let’s take a look at the open issues.

🐇 I think you have a more pressing problem. Your tests are failing. You need to fix them first before moving to another task.

🐢 Created a new issue. The logs say:

81Z error: 7 targets failed:
2022-10-29T18:18:36.8969467Z ##[error]    `-p xvc-workflow-tests --test test_storage_new_digital_ocean`
2022-10-29T18:18:36.8970697Z     `-p xvc-workflow-tests --test test_storage_new_gcp`
2022-10-29T18:18:36.8971279Z     `-p xvc-workflow-tests --test test_storage_new_generic_rsync`
2022-10-29T18:18:36.8971840Z     `-p xvc-workflow-tests --test test_storage_new_minio`
2022-10-29T18:18:36.8972387Z     `-p xvc-workflow-tests --test test_storage_new_r2`
2022-10-29T18:18:36.8973035Z     `-p xvc-workflow-tests --test test_storage_new_s3`
2022-10-29T18:18:36.8973577Z     `-p xvc-workflow-tests --test test_storage_new_wasabi`
2022-10-29T18:18:36.9368079Z ##[error]The process '/home/runner/.cargo/bin/cargo' failed with exit code 101

🐇 Looking at the logs, you should clean up those older branches.

🐢 Let’s do it now.

🐢 Done. I’ve deleted all branches except main.

🐇 Good. I think the best way is to delete them as soon as you merge them.

🐢 I’ve activated that setting. These were older branches.

🐇 Cool. Now, will you be checking the failing tests?

🐢 I think remote tests should never run if there are other errors. We can run coverage and remote tests as a second step after the first succeed.

🐇 That seems like a neat idea. You want to split the current job into two: one for compiling and testing the non-remote parts, and the second for coverage and storage tests. It won’t waste CI minutes that way?

🐢 On second thought, I think at the moment, that’s not a pressing issue. We should start fixing these tests at once. We can split them later.

🐇 Ok. It looks from the logs like secrets are not being made available to your jobs.

2022-10-29T18:18:17.5039311Z test test_storage_new_digital_ocean ... FAILED
2022-10-29T18:18:17.5039550Z
2022-10-29T18:18:17.5039652Z failures:
2022-10-29T18:18:17.5039784Z
2022-10-29T18:18:17.5040197Z ---- test_storage_new_digital_ocean stdout ----
2022-10-29T18:18:17.5040558Z Error: VarError { source: NotPresent }
2022-10-29T18:18:17.5040758Z
2022-10-29T18:18:17.5040765Z

🐢 I added them now. Let’s wait until the job ends to get a new set of logs.

🐇 You can write some documentation in the meantime.

🐢 I think #82 is a good candidate for this. There must not be too many missing docs in the ECS crate.

🐇 It was about walker and you fixed the ecs. You’re the most absentminded developer here, I believe.

🐢 Ooops, you’re right. I’ll add them together in a single PR.

🐇 In the meantime, the storage testing job has ended with 🔴.

🐢 Checking the raw logs. It looks like we didn’t update the tests to match the current options:

2022-10-30T14:45:50.1058348Z error: Found argument '--storage-prefix' which wasn't expected, or isn't valid in this context
2022-10-30T14:45:50.1059809Z ##[error]Found argument '--storage-prefix' which wasn't expected, or isn't valid in this context
2022-10-30T14:45:50.1061740Z 	If you tried to supply `--storage-prefix` as a value rather than a flag, use `-- --storage-prefix`
2022-10-30T14:45:50.1063220Z
2022-10-30T14:45:50.1063394Z USAGE:
2022-10-30T14:45:50.1064302Z     xvc storage new digital-ocean --name <NAME> --bucket-name <BUCKET_NAME> --region <REGION>

🐇 Is it --storage-prefix or --remote-prefix? Which one is clearer?

🐢 I think updating the tests to conform to the current options is better for now. We can update the options later if desired.

🐇 Another failure is rg. You assume it exists on the testing system.

🐢 ripgrep is available in Ubuntu 20.04, so we can just update the initial package list.

🐇 The same --storage-prefix failure appears in the minio tests.

🐢 Ok. Fixing it.

🐇 S3 tests want to run the new-s3 subcommand. I think we now see why we need these tests in the first place.

🐢 Yeah. Fixing the prefix option, too.

🐇 s3cmd is also required. You should add it, too.

🐢 Ok. I think we’ve added all missing dependencies. I’ll have to convert the mc tests for Minio to use s3cmd.

🐇 Then we can try again. Now, we can get back to documentation.

🐢 Added some more documentation to PR#93. I think it’s better to return to the storage test errors now.

🐇 It looks like you have missing DigitalOcean credentials.

🐢 Let’s take a look.

🐢 I’ve updated the tests to use a config file instead of command-line arguments that would be visible in the logs.

🐇 Good practice. You also need to remove previous logs.

🐢 Maybe there is an option for that.

🐇 It looks like there isn’t. There are masking options; when we add them to secrets, they are masked. But you shouldn’t use them in calls anyway.