Create HTTP PUT request to upload a file (c# & python)

透過 http 做 file upload 通常都是走 POST 比較多,但是最近遇到用 PUT 做 upload 的,需要自己生 multipart/form-data 的資料比較麻煩一點

整個 scenario 是這樣: 在同一個 session 裏先做 http post 做帳號密碼的檢查,然後取得 authentication key,然後再將該 key 放在下一個 put request header中,同時上傳檔案。

以C#來說,需要注意的就是要有 CookieContainer(),要自己組 multipart/form-data 的 header,還有就是資料形態要注意是 application/octet-stream

HttpWebRequest hwr;
CookieContainer mcc = new CookieContainer(); //create a CookieContainer, to keep session cookies

hwr = (HttpWebRequest)HttpWebRequest.Create("http://YOUR_URL/api/login");
hwr.Method = "POST";

var data = Encoding.ASCII.GetBytes("username=" + email + "&password=" + pwd);
hwr.ContentType = "application/x-www-form-urlencoded"; //post request should set ContentType as application/x-www-form-urlencoded
hwr.ContentLength = data.Length;
hwr.CookieContainer = mcc;

using (var stream = hwr.GetRequestStream())
{
    stream.Write(data, 0, data.Length);
}

string str_result;
using (var httpResponse = (HttpWebResponse)hwr.GetResponse())
{
    using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
    {
        var result = streamReader.ReadToEnd();
        str_result = result.ToString();
    }
}

dynamic tmp = JsonConvert.DeserializeObject(str_result); // in my example, request result is in JSON format, so I use JsonConvert

hwr = (HttpWebRequest)HttpWebRequest.Create("http://YOUR_URL/api/put/my");
hwr.CookieContainer = mcc;
hwr.Method = "PUT";
string str_boundary = "---WebKitFormBoundarySkAQdHysJKel8YBM";  // create the boundary
hwr.ContentType = "multipart/form-data; boundary=" + str_boundary;  // set http header ContentType as multipart/form-data, and specify the boundary
hwr.Headers.Add("Authorization", Convert.ToString(tmp.authHeader)); // set Authorization in header
hwr.Headers.Add("Cache-Control", "no-cache");
hwr.Headers.Add("Accept-Encoding", "gzip, deflate, sdch"); //set accept encoding
hwr.Headers.Add("Accept-Language", "zh-TW,zh;q=0.8,en-US;q=0.6,en;q=0.4");
hwr.Accept = "*/*";
hwr.KeepAlive = true;
hwr.AllowAutoRedirect = false;
hwr.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate; // set auto decompress

using (Stream memStream = new System.IO.MemoryStream())
{
    // NOTE: you should add "--" before the boundary, this is HTTP specification. Besides, end boundary needs "--" at the end.
    var boundarybytes = System.Text.Encoding.ASCII.GetBytes("--" + str_boundary + "\r\n");
    var endBoundaryBytes = System.Text.Encoding.ASCII.GetBytes("\r\n--" + str_boundary + "--");

    // Content-Disposition is form-data, and the content-type is application/octet-stream
    string headerTemplate = "Content-Disposition: form-data; name=\"{0}\"; filename=\"{1}\"\r\n" +
                            "Content-Type: application/octet-stream\r\n\r\n";

    string files = @"c:\myput.dat";

    memStream.Write(boundarybytes, 0, boundarybytes.Length);
    var header = string.Format(headerTemplate, "putDatas", "myput.dat");
    var headerbytes = System.Text.Encoding.ASCII.GetBytes(header);

    memStream.Write(headerbytes, 0, headerbytes.Length);

    using (var fileStream = new FileStream(files, FileMode.Open, FileAccess.Read))
    {
        var buffer = new byte[1024];
        var bytesRead = 0;
        while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
        {
            memStream.Write(buffer, 0, bytesRead);
        }
    }

    memStream.Write(endBoundaryBytes, 0, endBoundaryBytes.Length);
    hwr.ContentLength = memStream.Length;

    using (var requestStream = hwr.GetRequestStream())
    {
        memStream.Position = 0;
        byte[] tempBuffer = new byte[memStream.Length];
        memStream.Read(tempBuffer, 0, tempBuffer.Length);
        memStream.Close();
        requestStream.Write(tempBuffer, 0, tempBuffer.Length);
    }

    string result_2;
    var response = hwr.GetResponse();

    using (Stream stream2 = response.GetResponseStream())
    {
        byte[] tmp_b = new byte[10000];  // if you do not set auto decompress, the returned data is compressed
        stream2.Read(tmp_b, 0, 10000);

        result_2 = System.Text.Encoding.UTF8.GetString(tmp_b).Replace("\0", "");
    }
    hwr.Abort();

    response.Close();
    response.Dispose();

    memStream.Close();
}

以 python 來說,其實也差不多,我用了 requests 跟 MultipartEncoder,要不然會麻煩許多。

import requests
import json
import os
from requests_toolbelt.multipart.encoder import MultipartEncoder

filepath = 'C:\\myput.dat'
with open(filepath, mode='rb') as fh:
    filedata = fh.read()
    length = os.path.getsize(filepath)
    s = requests.session()
    
    response = s.post('http://YOUR_URL/api/login',
        json={"username":"user@email.com", "password":"pwd"}
    )
    print(response.content)
    print("\r\n")
    res_json = json.loads(response.content.decode())
    print(res_json['authHeader'])


    multipartdata = MultipartEncoder(
                    fields = {
                        'putDatas': ('myput.dat', open('C:\\myput.dat', 'rb'), 'application/octet-stream')
                    },
                    boundary = "---011000010111000001101001")


    
    url = "http://YOUR_URL/api/put/my"

    headers = {
        'Content-Type': "multipart/form-data; boundary=---011000010111000001101001",
        'Authorization': res_json['authHeader'],
        'Cache-Control': "no-cache",
        'Connection': 'keep-alive',
        'Accept': '*/*',
        'Accept-Encoding': 'gzip, deflate, sdch',
        'Accept-Language': 'zh-TW,zh;q=0.8,en-US;q=0.6,en;q=0.4',
        'postman-token': "14552a5c-1e22-7a74-4437-8e82124100ea"
        }

    response = s.put(url, data=multipartdata, headers=headers)

    print(response.content)
    print(response.headers)