This is my first question in here. Question is mostly about Python but a little bit about writing crypto trading bot. I need to calculate some indicators about trading like Bollinger Bands, RSI or etc in my bot.

Indicators are calculated from candlesticks data in trading. And candlesticks data updated in their time period. For example if use candlestiks data in period 5 minute i need to calculate indicators every 5 min (after candlesticks update), or if i use candlesticks data in period 1 hour period i need to calculate indicators every 1 hour.

So, think that i have two classes. One is named with "TradeControl" and it has a attribute named with "candlestick_data". And second class is named with "CandlestickUpdater". I want to manage updates for "candlestick_data" in "TradeControl" class from "CandlestickUpdater" class.

My problem is about parallel process (or threading) in python. I don't know how to do it proper way. Is there anybody here to give me some advice or sample code. And i am also wondering if i should use mutex-like structures to guard against conditions like race condition. What i am trying to do in code is like below. Also, if you have a different suggestion, I would like to consider it.

class TradeControl():
    def __init__(self):
        self.candlestick_data = 0

    def update_candlestick(self):
        """Candlesticks data will be requested from exchange and updated in here"""
        pass

    def run_trading_bot(self):
        while True:
            """Main trading algorithm is running and using self.candlestick_data in here"""
            pass
        pass

    pass

class CandlestickUpdater():
    def __init__(self, controller: TradeControl):
        self.controller = controller
             
    def run_candlestick_updater(self):
        while True:
            """Candlesticks data updates will be checked in here"""
            
            if need_to_update:
                """TradeControl will be informed in here"""
                self.controller.update_candlestick()
                pass
            pass
        pass
    pass

Thanks for your replies,

CodePudding user response：

It's not quite obvious from your description what CandlestickUpdater is for. It looks like its purpose is to notify TradeControl that time has come to update candlestick_data, whereas updating logic itself is in the TradeControl.update_candlestick method. So basically CandlestickUpdater is a timer.

There are different python libs from scheduling events. sched for example. It seems that it can be used for scheduling TradeControl.update_candlestick method with a required time period.

If you still want to have CandlestickUpdater class, I think it can actually work as it's written in your code. You can these classes in threads this way:

from threading import Thread

trade_control = TradeControl()
candlestick_updater = CandlestickUpdater(controller=trace_control)

Thread(target=trade_control.update_candlestick).start()
Thread(target=candlestick_updater.run_candlestick_updater).start()

Speaking of race condition. Well, it can be the case, but it depends on your implementation of update_candlestick and run_trading_bot. It's better to create TradeControl.lock: threading.Lock attribute and use it when updating candlestick_data: https://docs.python.org/3/library/threading.html#lock-objects.

There is also another way of communicating between threads without direct methods calls. Look at the blinker. It's nice lib to implement event-driven scenarios. You just need to create a "signal" object (outside of TradeControl and CandlestickUpdater classes) and connect TradeControl.update_candlestick method to it. Then you just call "signal.send" method in "CandlestickUpdater.run_candlestick_updater" and connected functions will be executed. It can be a little bit tricky to connect instance method to signal. But there are some solutions for it. The easiest one is to make CandlestickUpdater.run_candlestick_updater static if it's possible for you.

CodePudding user response：

Q :
_{" ... should (I) use mutex-like structures to guard against conditions like race condition "... ?}

A :
Given the facts, how the Python Interpreter works ( since ever & most probably forever, as Guido von Rossum has expressed himself ) with all its threads, distributing "a permission to work" for ~ 100 [ms] quanta of time, via central G-lobal I-nterpreter L-lock ( GIL-lock ), there are no race-conditions to meet in Python ( unless software-created live-locks or some indeed artificially generated mutually blocked obscurities ).

In other words, all Python Interpreter threads are principally collision-free, as GIL-lock re-[SERIAL]-ises any and all threads to one-and-only-ONE-works-(if-GIL-lock-owner)-ALL-others-WAIT...

Q :
_{" My problem is about parallel process (or threading) in python. "}

A :
Thread are never even "just"-[CONCURRENT] in Python, so never form a True-[PARALLEL] form of process-orchestration at all. Processes escape from the GIL-lock re-[SERIAL]-isation tyrany, yet, at many add-on costs and barriers from a shift into a new paradigm of distributed-processing

If Process-based [CONCURRENT]-processing
happens to get onto the scene :

If going into process-based "just"-[CONCURRENT]-processing ( using joblib.Parallel(...)delayed(...), multiprocessing or other modules ), there you start with as many independent ( each being inside still GIL-lock driven, but independently one from any other - an uncoordinated freedom of mutually independent GIL-locks' reservations ) processes, top-down copies ( all Win, some MacOS ) of the original Python Interpreter ( yes, due to RAM-footprint RAM-I/O it is [SPACE]- and [TIME]-domain expensive ).

Here you cannot enjoy calling local-process Class-method or similar mechanics for propagating state-changes in-process, as your "receiver" may reside in the original Python Interpreter process, which has principally separate address-space that your local-process cannot ( and shall never ) be allowed to touch, the less change a single bit thereof.

Yet we may use here tools for distributed-computing signalling and/or message-passing, like zeromq, pyzmq, nanomsg, pynng and the likes, that meet your needs, some of which have vast Quant/Fintech installed base with many language bindings/wrappers ( we use ZeroMQ since v2.11 among python AI/ML-Ensemble-of-Predictors, MQL4 for driving FX-Broker XTOs & MQL4 Backtesting simulation-farm, python non-blocking distributed logger, python remote CLI agents, scale-out C language tools for multiple systems' integration glue logic, signal-provisioning service, subscriber-management, ... macro-system health-monitor & alarm system )

The above sketched distributed-computing macro-system spans several platforms and has no problems to process sub-millisecond cadence of FX-market stream of QUOTE-events, handle tens of thousands managed-trade positions ( driving live FX-market XTO-modifications ) and maintain all QuantFX-technical indicator re-calculations on the fly.

Building one's own event-responsive MVC-framework -OR-
re-use a robust & stable & feature-rich one :

Having said the above, you may re-use another concept, when living in a comfort of plenty of time when going into just an H1-timeframe operations.

have o look on Tkinter.mainloop()-examples, with tools for both event-driven & timed-callbacks, that will permit you anything within the ceiling of your imagination, including GUI and infinitely many modes of Man-Machine-Interaction modes and gadgets.