binaryreader.ReadInt32() reads too big or -ve value

Question

I am working on client server application where server sends screen image to client using UDT protocol. below is my code server sends image using binary-writer and client receive data using binary-writer. Code works perfectly up to few iteration but after few iteration reader.ReadInt32() read number which is either too big or either negative. i have read most of the binary-reader related issue but not found any helpful.

 

Server Code:

public void SendImage(Bitmap bmp)
{
 BinaryWriter writer = new BinaryWriter(CStream);
  using (MemoryStream ms=new MemoryStream())
  {
     bmp.Save(ms, ImageFormat.Jpeg);//save to memory stream
     long fileSize = ms.Length;
     writer.Write(fileSize); //send length
     byte[] buffer = new byte[512 * 1024];
     long pos = 0;
     ms.Seek(0, SeekOrigin.Begin);
     while (pos < fileSize)
     {
        //If buffer size is smaller then filesize then send in parts
        Int32 toSend = buffer.Length < (fileSize - pos)? buffer.Length: (Int32)(fileSize - pos);
        ms.Read(buffer, 0, toSend); //read from memory stream to byte array
        writer.Write(toSend);
        ServerSocket.Send(buffer, 0, toSend);
        pos += toSend;
        Console.WriteLine(itration + ", fileSize : "+ fileSize + "," + toSend + ",[buffer]: " + buffer[toSend - 3] + "," + buffer[toSend - 2] + ", " + buffer[toSend-1]);
     }
   itration++;
   ms.Flush();
   ms.Close();
   }
}

Client Side Code:

private void startRead()
{
    while (true)
    {
        Bitmap inImage = ReceiveImage(Settings.Encrypted);//BitmapFromStream(Settings.Encrypted);
        theImage.Image = (Image)inImage;
    }
}
private Bitmap ReceiveImage(bool Encrypted)
{
   Bitmap img = null;
   byte[] buffer;
   BinaryReader reader = new BinaryReader(CStream);
   using(MemoryStream ms=new MemoryStream())
   {
      try
      {
         long size = reader.ReadInt64();//read file length first
         buffer = new byte[4 * 1024 * 1024];
         long read = 0;
         while (read < size)
         {
            int toRecv = reader.ReadInt32(); //here reader reads wrong value
            Console.WriteLine(itration + "filesize " + size + " Byte to read : " + toRecv);
            ReadFragment(reader, toRecv, buffer);
            ms.Write(buffer, 0, toRecv);
            read += toRecv;
         }
         itration++;
         ms.Position = 0;
         img = new Bitmap(ms);
       }
       catch (Exception Ex)
       {
                    Console.WriteLine(Ex.Message + " : " + Ex.StackTrace);
       }
       buffer = null;
       ms.Flush();
       ms.Close();
     }
    reader = null;
   return img;
}
private int ReadFragment(BinaryReader reader, int size, byte[] buffer)
{
  int read = 0;
  while (read < size)
  {
    read += reader.Read(buffer, read, size - read);
  }
  return read;
}

this is the console output from server

 

0, fileSize : 568633,524288,[buffer]: 0,162, 138

0, fileSize : 568633,44345,[buffer]: 31,255, 217

1, fileSize : 568468,524288,[buffer]: 2,138, 40

1, fileSize : 568468,44180,[buffer]: 3,255, 217

2, fileSize : 578806,524288,[buffer]: 198,177, 106

2, fileSize : 578806,54518,[buffer]: 1,255, 217

3, fileSize : 589967,524288,[buffer]: 20,81, 64

3, fileSize : 589967,65679,[buffer]: 15,255, 217

11, fileSize : 632653,524288,[buffer]: 243,255, 0

11, fileSize : 632653,108365,[buffer]: 0,255, 217

12, fileSize : 642562,524288,[buffer]: 205,20, 85

12, fileSize : 642562,118274,[buffer]: 7,255, 217

13, fileSize : 645552,524288,[buffer]: 236,49, 131

13, fileSize : 645552,121264,[buffer]: 0,255, 217

 

and console output from client

 

0 filesize 568633 Byte to read : 524288

0 filesize 568633 Byte to read : 44345

1 filesize 568468 Byte to read : 524288

1 filesize 568468 Byte to read : 44180

2 filesize 578806 Byte to read : 524288

2 filesize 578806 Byte to read : 54518

3 filesize 589967 Byte to read : 524288

3 filesize 589967 Byte to read : 65679

11 filesize 632653 Byte to read : 524288

11 filesize 632653 Byte to read : 108365

12 filesize 642562 Byte to read : 524288

12 filesize 642562 Byte to read : 118274

13 filesize 645552 Byte to read : 524288

13 filesize 645552 Byte to read : 671776930 <-- here it read either too big number or negative number Exception thrown: 'System.ArgumentException' in mscorlib.dll

 

I have tried binaryformater, protobuf instead of binaryreader but it still stops after few itration.help me find the cause.