How to upload more than 2 gb data in web service?

Question

Hi all,

I have a client and server application.

I want to allow user to add file upto 15 gb.

In web service, config file i did this setting. This is working for 2 gb.

I tried to increase the value of maxRequestLength property to upload file size for more than 2 gb.

But it is giving me error that
The value for the property 'maxRequestLength' is not valid. The error is: The value must be inside the range 0-2097151.

How can i upload file with more than 2 gb?

Please help..

Zoran Horvat · Answer

And here is the complete client: using System; using System.IO; using System.Windows.Forms; namespace UploadTest { class Program { [STAThread()] static void Main(string[] args) { OpenFileDialog ofd = new OpenFileDialog(); ofd.Title = "Select file to upload"; if (ofd.ShowDialog() == DialogResult.OK) { ServiceReference1.Service1SoapClient cli = new UploadTest.ServiceReference1.Service1SoapClient(); string fileHandle = null; using (System.IO.FileStream fs = new System.IO.FileStream(ofd.FileName, System.IO.FileMode.Open)) { byte[] buffer = new byte[4096]; fileHandle = cli.BeginFileUpload(); long currentPos = 0; using (BinaryReader sr = new BinaryReader(fs)) { int chunkLength = sr.Read(buffer, 0, buffer.Length); while (chunkLength > 0) { byte[] sendBuffer = buffer; if (chunkLength { sendBuffer = new byte[chunkLength]; Array.Copy(buffer, sendBuffer, chunkLength); } cli.UploadFilePart(fileHandle, sendBuffer, currentPos); currentPos += chunkLength; chunkLength = sr.Read(buffer, 0, buffer.Length); } Console.WriteLine("Uploaded file with handle: {0}", fileHandle); Console.WriteLine("Look under {0} for it.", Path.GetTempPath()); } } } Console.WriteLine("Finished... Press ENTER to continue..."); Console.ReadLine(); Console.WriteLine("Exiting..."); } } } Just one change, in the server create the file using Truncate file mode, because it's working with temp directory under which you don't have much control: FileStream fs = new FileStream(fi.FullName, FileMode.Truncate); I've corrected that line in the post above so that both client and server are ready to be used. Zoran

Deepika Chaudhary · Answer

Thanks zoran for your code..Much appreciated.

Zoran Horvat · Answer

Saving such a large file in one go on the file system would be a problem by itself. And again, chunking is a good way to go. Now, your suspicions made me actually write such Web service, just to show how simple it is. Here is the complete code: using System.IO; using System.Web.Services; namespace UploadFileService { /// /// Summary description for Service1 /// [WebService(Namespace = "http://tempuri.org/")] [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)] [System.ComponentModel.ToolboxItem(false)] // To allow this Web Service to be called from script, using ASP.NET AJAX, uncomment the following line. // [System.Web.Script.Services.ScriptService] public class Service1 : System.Web.Services.WebService { [WebMethod] public string BeginFileUpload() { FileInfo fi = new FileInfo(Path.GetTempFileName()); FileStream fs = new FileStream(fi.FullName, FileMode.Truncate); fs.Close(); // This will create the file return fi.Name; } [WebMethod] public void UploadFilePart(string handle, byte[] data, long appendAtPostion) { string filePath = Path.GetTempPath(); FileInfo fi = new FileInfo(filePath + handle); if (fi.Exists && fi.Length == appendAtPostion) { using (FileStream fs = new FileStream(fi.FullName, FileMode.Append)) fs.Write(data, 0, data.Length); } } } } As you can see it's something like 20 lines of code to implement chunked reception on Web service side. Any other solution would include Web server administration, settng up access rights, dealing with firewalls, etc. From this example you should learn that chunking is simple and effective method to deal with large amounts of data. Zoran

Deepika Chaudhary · Answer

In the config file..The folder path is mentioned..Inside it i am storing the received file.

Zoran Horvat · Answer

How do you store received file on the server? I have implemented chunked reception of files several times in the career and I guarantee you that it is very efficient and controllable solution. And not complicated at all to implement. Couple of functions grand total. Zoran

Deepika Chaudhary · Answer

I am trying to google more..to get some more idea of how to part the files. Is there any other way. Earlier the application was developed using ftp but now we are using web service. i have to do this in web service.

Zoran Horvat · Answer

Using sockets requires firewall settings that would allow incoming connections. FTP also requires specific setup. I know from practice that both methods are virtually impossible to use because network admins don't allow them for security reasons. Web services are typically applicable because they operate over the HTTP. You can't send viruses over HTTP because it is a text-based protocol. You can't break it down (easily) because it has implies active anti-DOS measures. Other methods don't have those benefits and hence often cannot be applied in real world cases. Zoran

Guest User · Answer

Is a web service the only choice here? I would use ftp or sockets for files this size.

Zoran Horvat · Answer

It would be difficult to produce sample code - it has to do a lot with design. However, I can give you directions how to design it. 1. Web service may have interface like this: ReceiveFilePart(byte[] part, long appendAt, long remainingLength) - this method would receive part of the file which should be written starting at specified position, and it also informs the server how much data remains to be sent to it. 2. Client iterates in a loop, reads parts of the file in predefined chunk sizes (e.g. 1 MB per step) and sends every part to the server in one call to ReceiveFilePart 3. Make sure that medium on which you write can accept such large files. 4. Make sure that client can cancel sending the rest of the file, e.g. by implemention session-like interface: int StartReceivingFile(long totalLength) - called before the first part is sent to the server; method returns integer which uniquely identifies the file which will be sent (like a kind of per-file session ID). void EndReceivingFile(int fileId, bool cancel) - completes sending of file with specified id previously returned by StartReceivingFile; cancel flag indicates whether client has given up sending the file or sending was complete. Hope this helps. Zoran

Deepika Chaudhary · Answer

Thanks for the reply. i have a one file. For example..1 doc file with 3 gb. Please can you give me some example or code? how to do upload this? bcoz i din't understand about creating web interface with part of files.

Zoran Horvat · Answer

maxRequestLength parameter is added to protect HTTP server from denial-of-service and similar attacks. If you need to upload significant amounts of data, then it is better to code such Web interface which receives parts of files. In that case your server can decide upon every particular request, whether to accept it or reject it, and it can receive large files as a series of smaller parts. This solution does not affect performance because sending gigabytes of data to Web service is a lengthy process anyway. Zoran