🐇 I think we can start by improving xvc.py. I mean, releasing. Yesterday we finished our work with the Rust library.

🐢 Then we can start to look at the GUI, I believe. We can replace that 3-column view with a table and preview. It will be much easier that way.

🐇 Yep. Let’s finish and release the Python version first. Then we’ll go on to the GUI.

🐢 Let’s take a look at the PRs first.

ghpl 
36	Bump pyo3 from 0.22.2 to 0.23.3	dependabot/cargo/pyo3-0.23.3	OPEN	2024-12-04T03:52:03Z

gh pr close 36
✓ Closed pull request iesahin/xvc.py#36 (Bump pyo3 from 0.22.2 to 0.23.3)

🐢 We have already upgraded to pyo3 0.23. No need for this. Let’s create a PR for the current branch.

git push --set-upstream origin v0.6.13

ghpC --fill
https://github.com/iesahin/xvc.py/pull/37

🐇 We can also tag and push the tags.

🐢 I’d like to have some more coverage for certain parts of the code. Let’s add some tests.

🐇 It looks like we mainly lack the storage tests. They need configuration to add keys to GitHub, and we already skip some of these even in the Rust code.

🐢 Umm, I see. We also need a way to measure coverage. Could we do this with Codecov, I wonder?

🐇 I found an example here: https://github.com/codecov/example-python/blob/main/.github/workflows/ci.yml. It needs coverage and pytest-cov in the requirements.

🐢 Let’s try this then.

🐇 Added coverage.yml file. It’s simpler than the other GitHub action. Need to update the token now.

🐢 There is an issue installing the requirements.

🐇 I forgot to add sudo to apt-get. Will take care of it now.

🐢 Let’s check the run.

ghrl
in_progress		Release v0.6.13	coverage	v0.6.13	pull_request	12556590550	2m31s	2024-12-31T06:59:58Z
in_progress		Release v0.6.13	publish-to-pypi	v0.6.13	pull_request	12556590548	2m31s	2024-12-31T06:59:58Z
...

🐇 It takes a while and we still didn’t add the CODECOV_TOKEN.

🐢 Let’s add it, and after that, we need to take a look at this blink configuration. It adds letters after the selection.

🐇 Added the secret and configured xvc.py for coverage. There is an error with the build, though. Maybe the command we should be using is maturin develop instead of build to make Xvc available for the environment.

🐢 Let’s update and try it then.

🐇 maturin develop requires a virtual environment.

🐢 I checked the options for build, and I think there is an option, but let’s search first.

🐇 I searched, but it looks like we can just pass an --out directory or install xvc from the target/wheels/ directory. The second option requires less maintenance.

🐢 Okay. Let’s add a step to the action then.

🐇 Now, let’s wait for the run to finish with gh run watch.

🐢 It failed. Let’s take a look at the logs:

ghrl
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12556939836	3m26s	2024-12-31T07:38:54Z

gh run view 12556939836 --log-failed

🐇 It looks like some of the tests are failing. Let’s run the tests locally.

🐢 We should run with the --forked option, and it looks like we have a test to update with the xvc file list.

🐇 Updated the test, and I noticed we forgot to supply the new --show-directories option in the Python interface.

🐢 Yep. Passing these options as command-line options in strings is not robust. It’s very easy to forget things. I think we should start using CLI structs directly, but it’s not time yet.

🐇 I agree. It’s one of the goals for building a GUI, actually.

🐢 The tests failed again.

ghrl | rg failure
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12557031091	3m49s	2024-12-31T07:51:19Z

gh run view 12557031091 --log-failed

🐇 There is a Git error now. We need to add a Git user and email to the action.

🐢 Added those and watching the results again now.

🐇 Why do you think the publish action always works? It can only run with the main branch, I think. No need to run it with other pushes.

🐢 Yes, let’s configure it now.

ghrl | rg failure
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12557083393	3m21s	2024-12-31T07:59:18Z

gh run view 12557083393 --log-failed

🐇 It looks like we also need xvc-test-helper in the path. Let’s cargo install it and add it to the path.

🐢 Added .cargo/bin to the path like:

- name: Add cargo bin to PATH
  run: echo "$HOME/.cargo/bin" >> $GITHUB_PATH

and installed the helper with cargo install xvc-test-helper.

🐇 By the way, GitHub Copilot is hallucinating about a method to update the path.

🐢 I searched and it may not be hallucinating. We can use the ::add-path:: command with echo, it looks like. This is new to me.

🐇 The tests failed again.

ghrl | rg failure
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12557208723	3m34s	2024-12-31T08:12:09Z

gh run view 12557208723 --log-failed

🐢 file().list() had a mistake, and we need to install rg for the tests.

🐇 Watching the test run. In the meantime, maybe we can review…

🐢 Failed again.

ghrl | rg failure
completed	failure	Release v0.6.13	coverage	v0.6.13	pull_request	12557294105	3m46s	2024-12-31T08:21:17Z

gh run view 12557294105 --log-failed

🐇 Added db.commit() to two places in the code. This should pass now.

🐢 Yeah!

ghrl | head -n 1
completed	success	Release v0.6.13	coverage	v0.6.13	pull_request	12557509836	3m46s	2024-12-31T08:44:33Z

gh run view 12557509836

✓ v0.6.13 coverage iesahin/xvc.py#37 · 12557509836
Triggered via pull_request about 4 minutes ago

JOBS
✓ linux in 3m36s (ID 35010253915)

ANNOTATIONS
! ubuntu-latest pipelines will use ubuntu-24.04 soon. For more details, see https://github.com/actions/runner-images/issues/10636
linux: .github#1


For more information about the job, try: gh run view --job=35010253915
View this run on GitHub: https://github.com/iesahin/xvc.py/actions/runs/12557509836

🐇 I can’t see a coverage report on codecov.io, though.

🐢 It says no coverage report is generated. Let’s test to generate XML files locally.

🐇 The option in the docs seems incorrect. --cov-branch doesn’t produce anything.

🐢 Let’s wait for the run again.

🐇 Should we add a badge to the README?

🐢 It won’t show much, but yeah, let’s make it.

🐇 The results are in and it shows 100% coverage. This means it doesn’t actually test anything.

🐢 We need Rust coverage for this. Let’s search for it.

🐇 I found this: https://github.com/cjermain/rust-python-coverage. It runs cargo llvm-cov with the project and measures test coverage. But we don’t have any Rust tests.

🐢 It looks like we don’t need Rust tests. cargo llvm-cov can check coverage with the Python as well.

$ cargo llvm-cov show-env --export-prefix
export RUSTFLAGS=" -C instrument-coverage --cfg coverage --cfg trybuild_no_target"
export LLVM_PROFILE_FILE="/home/.../rust-python-coverage/target/rust-python-coverage-%m.profraw"
export CARGO_INCREMENTAL="0"
export CARGO_LLVM_COV_TARGET_DIR="/home/.../rust-python-coverage/target"

🐢 Let’s take a look at what remained for 0.6.13.

🐇 I think there is nothing left. Python must be published when we merged the PR.

🐢 Let’s take a look by searching xvc python.

🐇 The PyPI page is https://pypi.org/project/xvc/ and it still reports the version as 0.6.11. There must be something.

🐢 Now let’s take a look at

https://github.com/iesahin/xvc.py

🐇 The run seems to be OK though.

https://github.com/iesahin/xvc.py/actions/runs/12557780141/job/35010938462

🐢 Maybe the version in pyproject.toml is still 0.6.11 and we forgot to update it?

tmux new-window -c $HOME/github.com/iesahin/xvc.py/ nvim

🐇 There is no version string in ~/github.com/iesahin/xvc.py/pyproject.toml. It’s dynamic.

🐢 Then, let’s try to publish from local now.

🐇 The iex username requires email verification.

🐢 It looks like I publish xvc through the iesahin account, not iex. Maybe I can add both accounts to the project.

🐇 There seem to be no errors on the PyPI site.

🐢 Updating the token. Let’s add the token to pass to run maturin publish.

🐇 We’re receiving invalid or non-existent authentication information. Upgraded maturin to see if it fixes the issue.

🐢 We can also double-check the key.

🐇 Uploaded successfully from local. Let’s check the job again.

🐢 The release job was skipped because we didn’t tag after the merge. https://github.com/iesahin/xvc.py/actions/runs/12557780141/job/35011303045 That looks like the reason.

🐇 I should be more careful which is run and which is skipped.

🐢 Maybe we can relax the condition. We do this rarely. Maybe republishing is alright?

🐇 Yep, removed that. The jobs are running now. Let’s watch them to see what happens when we publish some of the packages.

🐢 In the meantime, let’s experiment with searching commands file with fzf-lua.

🐇 It requires more experimentation, but we can start from https://github.com/ibhagwan/fzf-lua/wiki/Advanced#interactive-shell-command.

🐢 The xvc publish jobs failed, btw.

ghrl
completed	failure	Remove if condition from release	publish-to-pypi	v0.6.13	push	12568863716	15m26s	2025-01-01T08:30:03Z

gh run view 12568863716 --log-failed
...
Release	Run actions/download-artifact@v4.1.7	        Please ensure that your artifact is not expired and the artifact was uploaded using a compatible version of toolkit/upload-artifact.
...

🐇 It was using an older version of upload-artifact.

🐢 Rerunning the job and it looks like it runs for both push to main and tag with v0.6.13. We can turn off push to main, I believe.

🐇 There was a missing upload-artifact again. Fixed and updated the tags.

ghrl completed failure update upload-artifacts publish-to-pypi main push 12569060915 14m39s 2025-01-01T08:57:22Z

ghrf 12569060915

🐢 There are conflicts with uploaded artifacts now. We may need to clean up the artifacts manually.

🐇 The conflicts were not about inter-workflow names. The names were conflicting because all files were named wheel. I added the platform and target to the names to avoid conflicts.

🐢 Let’s wait then. Maybe it will work this time. Could you search for a Lua console for Neovim?

🐇 Let’s try this one: return { “yarospace/lua-console.nvim”, lazy = true, keys = “`”, opts = {}, }

🐢 I couldn’t make it run but won’t spend much time ATM. How about the jobs?

ghrl
completed	failure	update upload artifact names	publish-to-pypi	v0.6.13	push	12569298414	13m57s	2025-01-01T09:27:00Z

ghrf 12569298414
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9180510Z ##[group]Run actions/download-artifact@v4
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9182098Z with:
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9182859Z   name: wheels
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9183843Z   merge-multiple: false
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9184817Z   repository: iesahin/xvc.py
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9185820Z   run-id: 12569298414
Release	Run actions/download-artifact@v4	2025-01-01T09:40:52.9186903Z ##[endgroup]
Release	Run actions/download-artifact@v4	2025-01-01T09:40:53.1937617Z Downloading single artifact
Release	Run actions/download-artifact@v4	2025-01-01T09:40:53.4694769Z ##[error]Unable to download artifact(s): Artifact not found for name: wheels
Release	Run actions/download-artifact@v4	        Please ensure that your artifact is not expired and the artifact was uploaded using a compatible version of toolkit/upload-artifact.
Release	Run actions/download-artifact@v4	        For more information, visit the GitHub Artifacts FAQ: https://github.com/actions/toolkit/blob/main/packages/artifact/docs/faq.md

🐇 Now the download doesn’t work.

🐢 Update it to download using patterns.

🐇 Did so and let’s take a look at the jobs again.

ghrl
completed	failure	added pattern to download	.github/workflows/publish.yml	main	push	12569456859	0s	2025-01-01T09:49:19Z

ghrv 12569456859

X main .github/workflows/publish.yml · 12569456859
Triggered via push about 3 minutes ago

X This run likely failed because of a workflow file issue.

For more information, see: https://github.com/iesahin/xvc.py/actions/runs/12569456859

🐢 The line with the pattern was reported as broken.

🐇 Removed it and recommitted.

ghrl
✅ completed	success	remove pattern to download all	publish-to-pypi	v0.6.13	push	12569509435	14m41s	2025-01-01T09:58:49Z

🐢 And now this completes the v0.6.13 release.

🐇 We’ll see how it will work next time.

🐢 Yep. Let’s move on to the GUI for now. Is that okay with you?

🐇 It looks, from the coverage page, that our additions to HStore don’t have any tests. We can add some unit tests to new joins, maybe.

🐢 I don’t find unit tests particularly useful, but let’s use GitHub Copilot to add tests for us.

🐇 Added a unit test and a doc test for full_join, and I think doc tests have more value. They provide documentation, and we can readily see how to use a function from its docs. Better to increase coverage with doc tests.

🐢 There are points that I should learn while writing doc tests. The imports must use the full path, not crate::. The tested struct also doesn’t have implicit imports.

🐇 The ceremony of adding keys and values is a bit too much. It may be worthwhile to add insert for any Into<XvcEntity>.

🐢 It will certainly save time if we don’t have to type .into() for each key :)

🐇 Pushed to test again. Should we have some means to test coverage locally?

🐢 I don’t think we need to consider coverage locally. It’s not worth our time.

🐇 Now while waiting for tests to be completed, what can we do?

gh -R iesahin/xvc run list
completed	failure	v0.6.13	Rust-CI	v0.6.13	pull_request	12543831360	3m54s	2024-12-30T08:15:06Z
...

🐢 I don’t think it took too much time. Let’s view the results:

gh -R iesahin/xvc run view 12543831360
...
  X Run Current Dev Tests
...
To see what failed, try: 
View this run on GitHub: https://github.com/iesahin/xvc/actions/runs/12543831360

🐢 The current dev tests fail for some reason. Let’s run these locally.

🐇 We’re missing llvm-tools-preview locally. How do we install this?

🐢 The command is:

rustup component add llvm-tools-preview
info: component 'llvm-tools' for target 'aarch64-apple-darwin' is up to date

🐇 It’s already installed. We need to set the environment variables.

🐢 Instead, we can just turn off dev tests for the time being. We don’t need them. Our local tests pass.

🐇 Yeah, ok, we don’t need to solve each and every bit of these issues.

gh -R iesahin/xvc run list

… 🐇 Ok. Let’s take a look at the run again.

gh -R iesahin/xvc run list
completed	failure	v0.6.13	Rust-CI	v0.6.13	pull_request	12544020064	3m39s	2024-12-30T08:33:25Z

gh -R iesahin/xvc run view 12544020064
...
  X Test and Coverage
...

gh run view 12544020064 --log-failed
...
Test and Coverage (stable)	Test and Coverage	2024-12-30T08:36:58.8911690Z Error: ProcessError { stdout: "", stderr: "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to e1.xvc.dev port 80 after 160 ms: Couldn't connect to server\n" }
...

🐢 We need to start nginx on the server. We forgot it yesterday.

🐇 Ah, yeah. After adding that dufs installation. Ok.

🐢 We also need to add a reverse proxy to dufs somehow, but this is for later.

🐇 For the use case, I don’t think it’s necessary. We can just adjust the port to a non-standard one if we need to use 443 for another thing, but let’s take a look at the tests again.

🐢 Let’s add another doc test. This time to XvcStore.

cargo test -p xvc-ecs --doc
...
test result: ok. 8 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 2.87s

🐇 Sent the files again. Waiting for tests to finish.

🐢 Let’s check the keymaps file in the meantime.

🐇 I tried to read some documentation but didn’t see an error. Maybe we should just set it to non-lazy and avoid allowing remaps.

🐢 We already spent too much time with this.

🐇 Yes, let’s take a look at the tests again.

gh -R iesahin/xvc run list
completed	success	v0.6.13	Rust-CI	v0.6.13	pull_request	12544291068	8m10s	2024-12-30T09:00:42Z
...

🐢 Oh, yeah, the merge is ready.

🐇 Patch coverage is still behind the target, though.

🐢 Yeah, but let’s release this one and make the next better covered. Also, I’m not sure if the doc tests had any effect on coverage.

🐇 If we look at the coverage page again, we can see if the doc tests had any effect.

🐢 It seems codecov.io doesn’t consider coverage for doc tests. This is a bit weird, but let’s not spend more time on this.

🐇 Sure, let’s merge.

🐢 I think we forgot to bump the version in Cargo.toml. We’ll have to do that in main.

🐇 Oh, yeah. Let’s bump it and tag as well.

🐢 Now, we can wait for all files to be produced. What will we do next?

🐇 We can just release the Python version as well. It shouldn’t need any changes.

🐢 Umm, right. Maybe we can add a few tests as well.

🐇 Let’s bump the version first and see.

🐢 Bumped versions in Cargo.toml and ran maturin develop.

🐇 It seems ready now.

🐢 The main fails, though. The “Publish Crates” action looks for libsqlite3. Let’s take a look at it.

gh -R iesahin/xvc run list
completed	failure	Release v0.6.13	Publish Crates	v0.6.13	push	12544753359	6m1s	2024-12-30T09:41:56Z

🐢 The issue is that the VM doesn’t have libsqlite3-dev. Let’s add it.

🐇 We need to start the job manually again. Let’s not tag this time.

🐢 Some of the packages were already published. Now they break. crates.io says they’re already published. Maybe we can check if a package is already published.

🐇 Let’s check if we can make cargo publish more forgiving.

🐢 There doesn’t seem to be an option. Let’s search “how to skip published packages in the workspace to avoid errors with cargo publish”.

🐇 Claude is bullshitting again. Let’s try a manual approach: how to skip already published packages.

🐢 It may be easier to just add a check if the package is published. How do we get the info?

🐇 Or we can just go on to the next package if the package is already available.

🐢 Let’s do this manually this time.