ERROR: Cannot add 'folder-path', because it is overlapping with other DVC tracked output:-CodePudding

Goal: add commit push all contents of project_model/data/ to dvcstore.

I don't have any .dvc files in my project.

$ dvc add ./project_model/data/
ERROR: Cannot add '/home/me/PycharmProjects/project/project_model/data/images', because it is overlapping with other DVC tracked output: '/home/me/PycharmProjects/project/project_model/data'.
To include '/home/me/PycharmProjects/project/project_model/data/images' in '/home/me/PycharmProjects/project/project_model/data', run 'dvc commit project_model/data.dvc'

$ dvc commit project_model/data.dvc
ERROR: failed to commit project_model/data.dvc - 'project_model/data.dvc' does not exist

I've deleted contents from .dvc/cache/ and S3 s3://foo/bar/dvcstore/, with no luck.

$ dvc -V
2.10.2

$ dvc doctor
DVC version: 2.10.2 (pip)
---------------------------------
Platform: Python 3.9.12 on Linux-5.15.0-47-generic-x86_64-with-glibc2.35
Supports:
        webhdfs (fsspec = 2022.5.0),
        http (aiohttp = 3.8.1, aiohttp-retry = 2.5.2),
        https (aiohttp = 3.8.1, aiohttp-retry = 2.5.2),
        s3 (s3fs = 2022.5.0, boto3 = 1.21.21)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p5
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/nvme0n1p5
Repo: dvc, git

Please let me know if there's anything else I can add to post.

CodePudding user response：

Most likely you have /home/me/PycharmProjects/project/project_model/data defined as a stage output in your dvc.yaml:

stages:
  train:
  ...
  outs:
  - project_model/data

If that is the case, then you don't have to do an extra dvc add command to save this output to the remote storage. It should be taken care of by DVC automatically. Make sure that cache: false is not specified and that project_model/data is added to the .gitignore file though.

If you don't have dvc.yaml, but you are still getting this error, please try to run dvc dag --outs and dvc dag and share them with us.

CodePudding user response：

In my case, the problem was in dvc.yaml.

For a few stages, I had cyclical dependencies, where a file-path was mentioned in both the deps and outs.