Increase netcat read buffer from 4KB to 10KB-CodePudding

I try to use netcat as a quick-and-dirty socket server to receive requests with four words: m base exponent modulus, for example m 119911077684 3 25. I get back the result of the calculation.

This works OK until I send a request with a length of more than 4KB length.

My "server" script:

#!/bin/bash
eol=$'[^\n]*' # needed for bash regex

coproc nc -k -I 1 -l localhost 3000

while read -r input; do
  if [[ "$input" =~ ^m\ [0-9] \ [0-9] \ [0-9] ${eol} ]]; # m base exponent modulo
  then # modpow
    base=$(echo $input | awk {'print $2'})
    exp=$(echo $input | awk {'print $3'})
    mod=$(echo $input | awk {'print $4'})
    modPowResult=$(echo "(${base}^${exp})%${mod}" | bc | awk {'print $NF"."'})
    echo "Received modPow request for $input. Result: $modPowResult" 1>&2
    echo $modPowResult
  else
    echo $input | wc -c # if pattern does not match, echo byte length of request
  fi
done <&"${COPROC[0]}" >&"${COPROC[1]}"
kill "$COPROC_PID"

I start the script and send with another netcat instance the request:

$ echo "m 123456789 3 45" | nc -N localhost 3000
24.

This is correct. But when my request is much larger (see below, it is very long, ca. 10KB), I only get 4096 as reponse, because obviously my request gets cut off after 4KB and the pattern does not match anymore.

How can I increase the buffer size to 10KB?

I am using nc from the netcat-openbsd package, version 1-206-1ubuntu1 on Linux Mint 20.3

The long request that I send:

echo "m 119911077684388654464905913640776437154226670795608893126674401497793499190599280364130861725657104256566310462928507333209140730941832679850546964169574064502302506332980156346356353770204888295893093063184171997362057589517916010232980225391190320750952316465242324362891959100433858192271524147327414193082762890942035801958435316746500255772469345443216706039859433187598295446068294406788893004692563776136880999772523175303498038904595251052460892879542020390119365589551790081268294658670278839814000163620963699270901605327269182105315861334765282825877117237939020225059516048118822507748006320140239127478171838351066517019561839074929762330738028988275952662573265475737958301112851610595197531548497931446195952199321225970603280237392220002522234886604777354155305811126368542319814508734009724324574699762651933585658723268760338177467622997981508947488138668393644460194683162338182424638860784463205640320448417099383855382072818522929202040238994871533168133096831111510329881339134763052538793876982638221330371252613725855423571592146896758029880815370773533166968689127694703613734175707262846305586680102089283714672783584517247301712969997094188100846219871156880235250247262980941183982704470095357213637606107507347453851859858258698250607011513572535484062678271610484502455298006740369910804762417997008476244552290202422317383157478396506467483547744680169844974572967951447559272841442146195443163534430952926895944261405130537737083631290680916949035417377083444415613476470341303467658036432355832593865630222641297800142546121968228846670197431109658981524433553756516953169470270906175492638718257553629213095100747917452682253534211285832629838466511684445690038642779739911400231376076345271912866270126103787029556428130767593052626081831221831223069580448091335438929310705480858617960388692829148459668614553232856571359456182303111874697232026093026735470157165631651212560517025642976319831045781661538333599277034496981219269694512980275812089778730557632740797476353145760258328215998308472261145339422239050482532352129878920788865567995314629458455327155286460238383601946131961867426246265063123310726106693688718664044349515654363702741294452618335274329308578472175698617007182314284268398046497119531245936875755301603221798084316149598933360094885522867791011019273879574214058612225280983080451350767657126400778161555154304319817144617977921922780590457889389437733151154458594456669604913268357425018532675449321843724352306079286744292171948217434515478917308576498176270964164503811434926456683997368072955780726407859396433392794616507815229616209090535536963109780002140901827906975627967234293316955484216859007738529128975244401216085490915871531024507388309924318957155664906647334284585657638205005354547972419069669305452946639321947833394012435994802212916314476136632901669877608954569163544585174720274193743687733458152918021103753102103947757555536761250496423878947836539892181876319734828966403334468831996533582017308189424762055828357515602936391523294046491166211479541642683423998481465228274715001337998653123393277987627077776818291591865084393516551743127086475215375488202768688468221850610667101157756129583386649154755248232529575715947587573192188002157902017334436011038795700079402462826143560097658389200375725241631103202999992556312624592938137480846350848412026903419748096472380721662811550019522587811231052807795247261396319322540786904514244002649655444936516678315615815509490655441432557060506710854423809303803799832928358980279269976326443553767226117625378677721292146178144549137705670729846916993352305260256590882027591743518081622859024428432168499789050705296010312198688149480829077360824768339796901501769440011904610841999354713730007228733819400623210253248584255483381392135718982605469783073713176584693762824529856548255829134252116676403607113453428329546378911527121717218999162743029739470118529567736972067845106858183596445431859293814234467541362869837645245531687840718971450084815430300571277137631369480578785256511883434866398115621625953321395003577260895315118187411368327379978003552361264941991367588687121311204075732836962000799670126804000341688676901753702477021783266973371801241580953827616710188316672824198593419336050954281253130376778353822189824793800377718546601238247633120718659688831387817435649580877050448379012109503469868396520377378126968305945002734231194704115340864046955006030320932229997317614731989768411151227868361921843660344264921010915708495738985694574122911876672320122306870878930188687145058261733930157339319798025944904157368343854051187139367584319089952511053991867768078683402834031280370390040928451335081517037741108170426931092914661068970885818584596678595958636380828215961484655468488414470923291859914117819582725965617477747349015436236760229667348629664530116390566373561395785664859008004089692090065941440053939927994563495221580087238773509064814875447794256225 3 119911077684388654464905913640776437154226670795608893126674401497793499190599280364130861725657104256566310462928507333209140730941832679850546964169574064502302506332980156346356353770204888295893093063184171997362057589517916010232980225391190320750952316465242324362891959100433858192271524147327414193082762890942035801958435316746500255772469345443216706039859433187598295446068294406788893004692563776136880999772523175303498038904595251052460892879542020390119365589551790081268294658670278839814000163620963699270901605327269182105315861334765282825877117237939020225059516048118822507748006320140239127478171838351066517019561839074929762330738028988275952662573265475737958301112851610595197531548497931446195952199321225970603280237392220002522234886604777354155305811126368542319814508734009724324574699762651933585658723268760338177467622997981508947488138668393644460194683162338182424638860784463205640320448417099383855382072818522929202040238994871533168133096831111510329881339134763052538793876982638221330371252613725855423571592146896758029880815370773533166968689127694703613734175707262846305586680102089283714672783584517247301712969997094188100846219871156880235250247262980941183982704470095357213637606107507347453851859858258698250607011513572535484062678271610484502455298006740369910804762417997008476244552290202422317383157478396506467483547744680169844974572967951447559272841442146195443163534430952926895944261405130537737083631290680916949035417377083444415613476470341303467658036432355832593865630222641297800142546121968228846670197431109658981524433553756516953169470270906175492638718257553629213095100747917452682253534211285832629838466511684445690038642779739911400231376076345271912866270126103787029556428130767593052626081831221831223069580448091335438929310705480858617960388692829148459668614553232856571359456182303111874697232026093026735470157165631651212560517025642976319831045781661538333599277034496981219269694512980275812089778730557632740797476353145760258328215998308472261145339422239050482532352129878920788865567995314629458455327155286460238383601946131961867426246265063123310726106693688718664044349515654363702741294452618335274329308578472175698617007182314284268398046497119531245936875755301603221798084316149598933360094885522867791011019273879574214058612225280983080451350767657126400778161555154304319817144617977921922780590457889389437733151154458594456669604913268357425018532675449321843724352306079286744292171948217434515478917308576498176270964164503811434926456683997368072955780726407859396433392794616507815229616209090535536963109780002140901827906975627967234293316955484216859007738529128975244401216085490915871531024507388309924318957155664906647334284585657638205005354547972419069669305452946639321947833394012435994802212916314476136632901669877608954569163544585174720274193743687733458152918021103753102103947757555536761250496423878947836539892181876319734828966403334468831996533582017308189424762055828357515602936391523294046491166211479541642683423998481465228274715001337998653123393277987627077776818291591865084393516551743127086475215375488202768688468221850610667101157756129583386649154755248232529575715947587573192188002157902017334436011038795700079402462826143560097658389200375725241631103202999992556312624592938137480846350848412026903419748096472380721662811550019522587811231052807795247261396319322540786904514244002649655444936516678315615815509490655441432557060506710854423809303803799832928358980279269976326443553767226117625378677721292146178144549137705670729846916993352305260256590882027591743518081622859024428432168499789050705296010312198688149480829077360824768339796901501769440011904610841999354713730007228733819400623210253248584255483381392135718982605469783073713176584693762824529856548255829134252116676403607113453428329546378911527121717218999162743029739470118529567736972067845106858183596445431859293814234467541362869837645245531687840718971450084815430300571277137631369480578785256511883434866398115621625953321395003577260895315118187411368327379978003552361264941991367588687121311204075732836962000799670126804000341688676901753702477021783266973371801241580953827616710188316672824198593419336050954281253130376778353822189824793800377718546601238247633120718659688831387817435649580877050448379012109503469868396520377378126968305945002734231194704115340864046955006030320932229997317614731989768411151227868361921843660344264921010915708495738985694574122911876672320122306870878930188687145058261733930157339319798025944904157368343854051187139367584319089952511053991867768078683402834031280370390040928451335081517037741108170426931092914661068970885818584596678595958636380828215961484655468488414470923291859914117819582725965617477747349015436236760229667348629664530116390566373561395785664859008004089692090065941440053939927994563495221580087238773509064814875447794256225" | nc -N localhost 3000

CodePudding user response：

This is the man page for netcat. https://docs.oracle.com/cd/E86824_01/html/E54763/netcat-1.html The -b switch should allow you to increase your buffer size. Default is 1024, but it could be different for different distributions. Edit: Ok, this post shows the use of stdbuf. Force netcat to send messages immediately (without buffering) Sorry for any misunderstanding. Edit: There’s a command called socat which might help. Here is the man page. https://linux.die.net/man/1/socat

Edit: It seems like the buffer is written into the source code.

iPhone:~/netcat-0.7.1/src# grep -Hn "4096" *
misc.c:264:  char buf[4096], *p, *rest;
iPhone:~/netcat-0.7.1/src# grep -Hn "1024" *
core.c:185:      unsigned char buf[1024];
core.c:287:     /* FIXME: this ONLY saves the first 1024 bytes! and the others? */
core.c:528:  unsigned char buf[1024];
misc.c:347:        "Example: '1-1024'\n"));

I guess it would be possible to edit it and use the buffer size you want, but I’m not sure if there are any caveats.

CodePudding user response：

Quoting the variables fixed the issue.

The solution:

eol=$'[^\n]*' # needed for bash regex
if [[ "$input" =~ ^m\ [0-9] \ [0-9] \ [0-9] ${eol} ]];
    then
      base=$(echo "$input" | awk {'print $2'})
      exp=$(echo "$input" | awk {'print $3'})
      mod=$(echo "$input" | awk {'print $4'})
      modPowResult=$(echo "(${base}^${exp})%${mod}" | (export BC_LINE_LENGTH=0; bc))
      printf "%s." "$modPowResult" # echo would work here, too
fi

Cyrus' comment set me on track to solve the issue. I don't know enough about bash to understand it, but when quoting the $input throughout the script solved my issue.

The topics that came up when I searched for these kind of things (stdbuf, socat, bash/kernel buffer settings etc., also mentioned in this answer) where not the culprit here.