I've written some code to break up independent chunks of a computation into pieces and then feed each of the pieces to a thread pool which then does the computation and passes the result back the main thread via a mpsc.

The code is as follows:

pub fn impute(pbwt_data : Arc<&pbwt::PbwtInfo>, test_sequences : Vec<Vec<u8>>) -> Vec<Vec<u8>> {

    let N = test_sequences.len();
    let mut fin_imputed: Vec<Vec<u8>> = vec![Vec::new();N];
    let safe_pbwt = Arc::new(pbwt_data);

    let (tx,rx) = mpsc::channel();
    let pool = ThreadPool::new(8);
    let mut i = 0;

    for tst_seq in test_sequences.into_iter(){

        let txr = tx.clone();
        let safe_test_seq = Arc::new(tst_seq);
        let refer = Arc::clone(&safe_pbwt);

        let clo = closure!(move txr, move refer, move safe_test_seq, || {
            let val = impute_single(&refer,&safe_test_seq);
            let ret = RowWPos { position: i as u32, row: val};
            txr.send(ret).unwrap();
        });


        pool.execute(clo);
        i  = 1;

    }

    drop(tx);
    for recd in rx.iter() {
        fin_imputed[recd.position as usize] = recd.row;
    }

    return fin_imputed;
}

The inner impute_single function takes a reference to a PbwtInfo object and a reference to a Vec<u8> object. The inner function does not modify the PbwtInfo object in any way, it just reads from it.

Unfortunately when I try to run it this code produces an error:

error[E0521]: borrowed data escapes outside of function
   --> pbwt_hkhan/src/imputer.rs:138:9
    |
111 | pub fn impute(pbwt_data : Arc<&pbwt::PbwtInfo>, test_sequences : Vec<Vec<u8>>) -> Vec<Vec<u8>> {
    |               ---------       - let's call the lifetime of this reference `'1`
    |               |
    |               `pbwt_data` is a reference that is only valid in the function body
...
138 |         pool.execute(clo);
    |         ^^^^^^^^^^^^^^^^^
    |         |
    |         `pbwt_data` escapes the function body here
    |         argument requires that `'1` must outlive `'static`

For more information about this error, try `rustc --explain E0521`.
error: could not compile `pbwt_hkhan` due to previous error

I'm able to fix this error by changing the pbwt_data : Arc<&pbwt::PbwtInfo> in the function signature to pbwt_data : Arc<pbwt::PbwtInfo> and creating the Arc with the PbwtInfo object directly rather than with a reference to it before I feed it to the function. However this means my object is now stuck inside of the Arc and there is further code downstream where I'd like to use the PbwtInfo object again.

I could just clone it but it's a pretty big and expensive object to create a copy of so I'd rather not do this. Ideally I'd like a solution where I don't even have to pass an Arc to the impute function and can get away with just a &PbwtInfo as nothing it or anything it calls ever modifies pbwt_data. I know I can do this if instead of splitting off the computation into chunks and using a threadpool I just use a single thread so I feel like it should be possible here.

CodePudding user response：

Use Arc<pbwt::PbwtInfo> as type. To retreive the object again you can use Arc::try_unwrap(your_arc) which will give the object if there is only 1 strong reference left to it.