404/405 results using HttpClient in c# to upload/rename/delete files on a web server

by

Summary

This postmortem addresses a common architectural misunderstanding when migrating from legacy WebClient/WebRequest APIs to HttpClient in .NET 9. The developer encountered 404 (Not Found) and 405 (Method Not Allowed) errors while attempting to perform file operations (Upload, Rename, Delete) on an IIS 10 web server.

The core issue is not a code syntax error, but a protocol mismatch: HttpClient defaults to RESTful HTTP semantics (PUT/DELETE/POST), whereas the target IIS environment expects WebDAV operations to access remote file system features (rename/delete) and requires specific directory URL formatting for file uploads. The legacy WebClient masked these details by implicitly handling WebDAV verbs and URL formatting, which HttpClient exposes directly.

Root Cause

The errors stem from the interaction between the client-side HttpClient implementation and the server-side IIS configuration, specifically:

  1. WebDAV Dependence for File Operations:

    • Rename (404 Not Found): The server requires a WebDAV MOVE or COPY verb (often accessed via a specific collection URL). Sending a standard POST to a resource path that isn’t configured as a WebDAV collection endpoint results in a 404.
    • Delete (405 Method Not Allowed): IIS static file handlers generally do not accept DELETE verbs for arbitrary files. WebDAV must be installed and enabled on the server to handle DELETE requests for remote files. If WebDAV is not enabled, IIS returns 405.

  2. Incorrect Endpoint URL for Uploads:

    • Upload (405 Method Not Allowed): When HttpClient posts MultipartFormDataContent to a folder URL (e.g., http://server/files/), it does not automatically append the filename to the URL. It sends the request to the folder. If the server’s WebDAV module expects the target URL to be the full path of the new file (e.g., http://server/files/newfile.xml) to create it, posting to the directory can result in a 405 or 409 (Conflict).

  3. Missing Authorization Header:

    • While the HttpClientHandler is configured with NetworkCredential, IIS (when not using Anonymous Authentication) often expects a Authorization: Basic ... header for basic auth. Depending on the server configuration and the security module, credentials set in the handler might not be sent automatically for non-standard verbs (like WebDAV) unless explicitly forced or if the server challenges correctly.

Why This Happens in Real Systems

  • Legacy Abstraction Leakage: WebClient abstracted away the underlying protocol differences between standard HTTP and WebDAV. Migrating to HttpClient removes this abstraction, forcing the engineer to understand the specific HTTP verbs and URL structures the server expects.
  • Server-Side Configuration Dependencies: The behavior of PUT, DELETE, and MKCOL (Make Collection) is strictly controlled by IIS modules. If WebDAV is not explicitly enabled in IIS features, or if the web.config handlers are strict, standard HttpClient requests will be rejected.
  • Ambiguous Naming: Using remoteFolder as a variable name for the destination of a file upload is misleading. In HTTP, a folder is a collection resource; a file is a member resource. Posting to the collection requires the server to accept member creation via POST, which is not standard IIS behavior (IIS prefers PUT for file writes).

Real-World Impact

  • Migration Blockage: The project is stuck in a proof-of-concept phase because critical file management features are non-functional.
  • Inefficient Troubleshooting: The developer is “pulling hair out” chasing generic HttpClient examples on Stack Overflow that assume RESTful APIs, not WebDAV file system access.
  • Security Risks: Improperly handling credentials (e.g., attempting to force Basic auth without TLS/SSL) or constructing URLs via string concatenation (origName + "/rename") introduces risks of injection or credential leakage.
  • Performance Overhead: Creating HttpClient instances per request (if not done correctly) leads to socket exhaustion, though the provided code snippet shows correct instantiation.

Example or Code

To successfully perform these operations on an IIS WebDAV folder, you must use the specific HTTP verbs supported by the WebDAV protocol.

1. Renaming a File (WebDAV MOVE)

Instead of POST (which is ambiguous), use the MOVE verb.

private async Task RenameWebFile(string sourceUrl, string destinationUrl)
{
    // Ensure destination URL is absolute
    var request = new HttpRequestMessage(HttpMethod.Get, sourceUrl);
    request.Method = new HttpMethod("MOVE");
    request.Headers.Add("Destination", destinationUrl);

    // Force overwrite if required
    request.Headers.Add("Overwrite", "T"); 

    var response = await _client.SendAsync(request);
    return response.IsSuccessStatusCode;
}

2. Deleting a File (WebDAV DELETE)

Ensure HttpMethod.Delete is used, but verify the URL points to the specific file resource.

private async Task DeleteWebFile(string fileUrl)
{
    var request = new HttpRequestMessage(HttpMethod.Delete, fileUrl);
    var response = await _client.SendAsync(request);
    return response.IsSuccessStatusCode;
}

3. Uploading a File (WebDAV PUT)

Standard IIS WebDAV expects PUT requests sent to the full URL of the new file.

private async Task UploadFile(string localFilePath, string remoteFileUrl)
{
    using (var fileStream = File.OpenRead(localFilePath))
    {
        var content = new StreamContent(fileStream);
        content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");

        var request = new HttpRequestMessage(HttpMethod.Put, remoteFileUrl)
        {
            Content = content
        };

        var response = await _client.SendAsync(request);
        return response.IsSuccessStatusCode;
    }
}

How Senior Engineers Fix It

  1. Verify Server Capabilities: Before rewriting client code, inspect the IIS server. Confirm WebDAV Authoring Rules are installed and enabled. Use tools like Fiddler or Wireshark to see what HTTP verbs the server actually accepts.
  2. Use Correct Verbs:
    • Upload: PUT to the full file URL.
    • Rename: MOVE with a Destination header.
    • Delete: DELETE to the full file URL.
  3. Adjust URL Construction: Change the logic from “posting to a folder” to “putting to a resource.”
    • Bad: POST /files/
    • Good: PUT /files/newfile.txt
  4. Enforce Authentication: If NetworkCredential fails, explicitly set the Authorization header manually (for Basic Auth) or switch to Bearer tokens if the server supports modern auth.
  5. Implement Retry Logic: WebDAV operations can be flaky over poor networks. Senior engineers wrap these calls in exponential backoff policies.

Why Juniors Miss It

  • Assumption of Standardization: Junior engineers often assume all file servers speak the same HTTP language. They treat a WebDAV server like a simple REST API endpoint (using POST for everything) rather than a file system abstraction.
  • Over-reliance on Copy-Paste: Searching for “C# upload file to server” yields MultipartFormDataContent examples. These work for web API forms but fail for static file WebDAV servers where a raw byte stream PUT is required.
  • Confusion of Resources: In HTTP, a folder is a resource, and a file is a resource. Juniors often struggle to conceptualize that uploading a file requires creating a new resource URL, not just adding a payload to an existing container URL.
  • Debugging Blindly: Without tools like Fiddler to inspect the actual HTTP request/response, juniors guess at whether the issue is authentication, URL, or method, leading to scattered attempts.