Upload binary files using python-gitlab API-CodePudding

I'm tasked with migrating repos to gitlab and I decided to automate the process using python-gitlab. Everything works fine except for binary or considered-binary files like compiled object files ( .o ) or .zip files. (I know that repositories are not place for binaries. I work with what I got and what I'm told to do.)

I'm able to upload them using:

import gitlab

project = gitlab.Gitlab("git_adress", "TOKEN")

bin_content = base64.b64encode(open("my_file.o", 'rb').read() ).decode()

and then:

data = {'branch':'main', 'commit_message':'go away', 'actions':[{'action': 'create', 'file_path': "my_file.o", 'content': bin_content, 'encode' : 'base64'}]}

project.commits.create(data)

Problem is that content of such files inside gitlab repository is something like:

f0VMRgIBAQAAAAAAAAAAAAEAPgABAAAAAAAAAAAAA....

Which is not what I want. If I don't .decode() I get error saying:

TypeError: Object of type bytes is not JSON serializable

Which is expected since I sent file opened in binary mode and encoded with base64.

I'd like to have such files uploaded/stored like when I upload them using web GUI "upload file" option.

Is it possible to achieve this using python-gitlab API ? If so, how?

CodePudding user response：

The problem is that Python's base64.b64encode function will provide you with a bytes object, but REST APIs (specifically, JSON serialization) want strings. Also the argument you want is encoding not encode.

Here's the full example to use:

from base64 import b64encode
import gitlab
GITLAB_HOST = 'https://gitlab.com'
TOKEN = 'YOUR API KEY'
PROJECT_ID = 123 # your project ID
gl = gitlab.Gitlab(GITLAB_HOST, private_token=TOKEN)
project = gl.projects.get(PROJECT_ID)

with open('myfile.o', 'rb') as f:
    bin_content = f.read()
b64_content = b64encode(bin_content).decode('utf-8')
# b64_content must be a string!

f = project.files.create({'file_path': 'my_file.o',
                          'branch': 'main',
                          'content': b64_content,
                          'author_email': '[email protected]',
                          'author_name': 'yourname',
                          'encoding': 'base64',  # important!
                          'commit_message': 'Create testfile'})

Then in the UI, you will see GitLab has properly recognized the contents as binary, rather than text: