Networking part BinaryPacket behavior

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Networking part BinaryPacket behavior

      Hi guys,

      Your book is awesome. I just started to code the networking part, small client /server to test your code. I have a pitfall and I don't understand why you don't have it. That's some times I haven't write c++ code so I hope my problem is not too obvious.

      Here we go.

      When I debug my small project that is using BinaryPacket, m_Data is always empty.

      Here is your code snippet from your code website =

      Source Code

      1. ...
      2. char *m_Data;
      3. ...
      4. inline BinaryPacket::BinaryPacket(char const * const data, u_long size)
      5. {
      6. m_Data = GCC_NEW char[size + sizeof(u_long)];
      7. GCC_ASSERT(m_Data);
      8. *(u_long *)m_Data = htonl(size+sizeof(u_long));
      9. memcpy(m_Data+sizeof(u_long), data, size);
      10. }
      Display All


      This line is my problem =

      *(u_long *)m_Data = htonl(size+sizeof(u_long));

      Either one of the four bytes of the long is 0x00 (That will be more likely to be the case...), that will end the character sequence "m_Data". End of the story.(For me). And confirmed by VS debugger. m_Data is always null terminated after this line.
      Say I have a size of 5(+4 sizeof(u_long)), htonl will send back 0x09000000, first byte of the long is 0x00 that is put in the first m_Data memory space. If any of those bytes aren't 0x00, that works of course.

      How do you guys convert a long into a char sequence without having this null terminated behavior ?
      Either I don't see something big in front of me or a specific configuration ? Logically that makes sense so I know my outdated C++ knowledge is the key.

      I found this topic related to it, they don't seem to have the problem but I still do when I try
      stackoverflow.com/questions/54…onvert-long-to-char-const

      I looked your AttachRemotePlayer method for a BinaryPacket example. But whatever, the problem comes with the size, not the data.

      What did I miss ?

      Thank you and keep doing your great work.
    • In this case, null termination does not matter, in fact the entire point of the extra '+ sizeof(u_long)' is so that we can properly load the entire binary buffer without worrying about null termination, let me break this down a bit.
      • Client creates a binary packet with data of size 16
      • We allocate an internal buffer for the data to be copied, but also give ourselves an extra 'header' size of 4 bytes (sizeof(u_long))
      • We set the first 4 bytes to the size of the entire buffer, this however may be of a different endianess once it's crossed the network, which htonl converts for us.
      • We than copy the provided data to the buffer, but after our header.
      • On the client side, it would be up to the network handler to do something like this

      Source Code

      1. void NetworkPacketHandler(BinaryPacket* pPacket)
      2. {
      3. assert(pPacket);
      4. if(pPacket)
      5. {
      6. u_long nPacketSize = ntohl(*(u_long*)pPacket->m_Data);
      7. InterperateBinarySomehow(pPacket->m_Data + sizeof(u_long), nPacketSize - sizeof(u_long));
      8. }
      9. }
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz
    • Thanks for the answer. Yeah I figured that out so maybe I don't understand what you try to tell me.

      We set the first 4 bytes to the size of the entire buffer, this however may be of a different endianess once it's crossed the network, which htonl converts for us.


      Here is the thing.
      Step by step debugging, before the line

      Source Code

      1. *(u_long *)m_Data = htonl(size+sizeof(u_long));


      in the watcher = m_Data has memory initialized with whatever.

      after the line

      Source Code

      1. *(u_long *)m_Data = htonl(size+sizeof(u_long));


      in the watcher = m_Data is = /0 (because I have 0x09000000 so first byte is 0x00, so null).
      After

      Source Code

      1. memcpy(m_Data+sizeof(u_long), data, size);

      in the watcher m_Data still = /0

      If I force the unsigned long say to 0x09090909, that will work as m_Data has no null terminated in the first 4 bytes.
      And then I see the message on the server side.

      Does that make more sense ? Or maybe you can focus on how an unsigned long is set inside m_Data. How should it look or does it look in your case in memory after the htonl function ?

      Thanks !
    • I'll break down the operations for you
      • *(u_long*)m_Data : This is giving you a reference to the front of the 'm_Data' buffer, except it is being interpreted as a u_long, any operations you perform on this reference will treat that location in memory accordingly, it is ok for m_Data to have '/0' because it is just part of the u_long, which in this case is 9 bytes long.
      • On the receiving side of the network packet, the process is simply reversed. u_long nPacketSize = ntohl(*(u_long*)pPacket->m_Data); This will take the first 4 bytes of m_Data, and interpret it as a u_long again, which we than convert to the host endianess, giving us 9 bytes on the host side.
      • With this info we can than do whatever we like, a simple case is a memcpy, ie. memcpy(pBuffer, m_Data, sizeof(char) * nPacketSize);
      For reference, when converting from host to network on windows, you will get little endian to big endian. Which will be a conversion of 0x00000009 to 0x09000000. When you convert back it will simply reverse this (assuming it is a little endian system you are receiving the packet on).
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz
    • I really appreciate the time you take about my query. Though I know your explanation already.
      I will focus on your sentence. And just focus on the client side because I don't need more.


      *(u_long*)m_Data : This is giving you a reference to the front of the 'm_Data' buffer, except it is being interpreted as a u_long, any operations you perform on this reference will treat that location in memory accordingly, it is ok for m_Data to have '/0' because it is just part of the u_long, which in this case is 9 bytes long.


      Exactly, and that's what I expected ! It is not the result I have when I debug the function. I think what I need is really to see this line working for real from an external source.
      For me the translation of this line is = Store the four bytes of the u_long in the first 4 char of m_Data. Great but m_Data is still a char array so if there is a null terminated character, m_Data will be null. And that's what I get.

      Again here is what I have in the watcher when I debug this function in VS =

      for m_long = 0x09090909, after htonl line
      m_Data = \x09\x09\x09\x09 => That's good ! 4 Bytes init
      Then I see the next line working =
      memcpy(m_Data+sizeof(u_long), data, size);

      for m_long = 0x09000000, after htonl line
      m_Data = \0 => null terminated char array.
      Next line does nothing because of the previous null terminated.
      memcpy(m_Data+sizeof(u_long), data, size);

      Again a big thank you for your help and your time, the logical way I get it but for real that doesn't work for me and that's what I am trying to understand. A proof of the logical explanation is what I need.(That at runtime in the watcher m_Data can have a sequence like m_Data = /x09/x00/x00/x00abcd).

      Thanks !
    • Next line does nothing because of the previous null terminated.memcpy(m_Data+sizeof(u_long), data, size);


      This is not true, char arrays can have null terminations, character strings cannot. Both use the same storage (char*) however the interpretation is different. A function like 'strcmp' will search until it finds the null termination, a function like memcpy will not as you have supplied a size, the memory pointed to in this case points past the initial 4 bytes anyway so they are not even considered.
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz
    • I agree with you again that what I expected. I assumed the sentence by what I see in the watcher.
      So far so good, I will make code snippets to demonstrate what I have.
      I still don't understand why the watcher show me a null terminated string in m_Data after the htonl affectation. It is like it interprets m_Data as a string.
    • Well of course that works with a simple code sample in a simple online editor. X(
      Now I really guess this comes from the visual studio interpretation of m_Data and the fact I don't know very well that editor.
      Thank you again for your answers. I guess that will be enough for me to get through this. At least your posts confirm it should work the way it should. :)
    • No problem, let us know if you need any more help.

      I still don't understand why the watcher show me a null terminated string in m_Data after the htonl affectation. It is like it interprets m_Data as a string.


      This is because m_Data is still a char string, the watch window will still display it as such, so it will show a '\0' in the watch window. However the data is interpreted as a raw binary stream stored in the m_Data, so you can treat it however you like. If you want to see a cool use of the watch window in this case, make a new watch while you have a breakpoint after you set the size in m_Data, make the watch name *(u_long*)m_Data , this is how the watch window will know how to interpret your data.
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz