Bitcoin Stratum shares rejected despite successful submission — MacMetal Miner on Apple Silicon

by

Summary

A Bitcoin miner on Apple Silicon using the Metal framework was submitting shares that the pool rejected with a diff mismatch or low difficulty error, despite the local log showing a difficulty of 38. The root cause was a difficulty target mismatch: the miner was correctly hashing at the pool-assigned difficulty, but the pool’s difficulty retarget algorithm (specifically in Stratum v1’s vardiff implementation) failed to update the miner’s assigned target, or the miner failed to persist and use the dynamic difficulty assigned by the pool, leading to a stale share or invalid difficulty scenario relative to the pool’s current expectation.

Root Cause

The primary root cause was the lack of persistence in handling Stratum difficulty changes. In Stratum v1, the pool can dynamically adjust the difficulty (VarDiff) sent via a mining.set_difficulty message. The miner must store this value and apply it to all subsequent shares.

The specific failure chain for this Metal miner was:

  • State Management Failure: The Metal shader/kernel logic or the host-side Objective-C/Swift dispatch code likely reset or ignored the difficulty value received from the pool after the initial connection.
  • Normalization Error: The local “Diff: 38” log entry suggests the miner might be hardcoded or calculating a static difficulty rather than using the pool’s assigned target.
  • Vardiff Latency: The pool assigned a lower difficulty (e.g., Diff 1 or 8) to manage latency, but the miner continued submitting solutions calculated at the initial high difficulty (or vice versa), causing a diff mismatch.

Why This Happens in Real Systems

  • Stratum Vardiff Asynchronicity: Pools constantly monitor share response times and send mining.set_difficulty messages asynchronously. If the miner implementation treats the connection as “set and forget,” it desynchronizes from the pool.
  • GPU Pipeline Latency: Metal compute shaders operate on a massive parallel queue. If the difficulty changes, the CPU host must synchronize the new target with the GPU kernel arguments immediately. If the GPU queue is deep (high throughput), stale shares (calculated against old difficulty) might still be in flight.
  • JSON-RPC ID Collisions: If the miner uses a non-unique or static ID for mining.submit, the pool might reject the submission logically, but the miner misinterprets the response.

Real-World Impact

  • Zero Revenue: The miner consumes significant power and thermal budget (M3 Pro GPU) but earns zero Bitcoin because accepted shares = 0.
  • Pool Bans: Repeated invalid submissions can trigger IP bans or worker bans on the pool side.
  • Diagnostic Noise: The “Diff: 38” log entry is misleading, masking the fact that the pool likely requested a different target (e.g., Diff 256 or Diff 1).

Example or Code

The Stratum protocol relies on the mining.set_difficulty message to define the target. The miner must convert the pool’s difficulty to a target (target = (2^256 / difficulty)) for the hash check.

Here is a Python simulation of the target calculation logic that the Metal kernel logic should mimic, specifically handling the dynamic difficulty update:

import hashlib
import struct

def difficulty_to_target(difficulty):
    """
    Converts a Stratum difficulty (float) to a 256-bit target integer.
    Target = (2^256) / difficulty
    """
    # 2^256
    max_target = (1 << 256)

    if difficulty == 0:
        return max_target

    target = max_target // int(difficulty)
    return target

def check_share(share_data, pool_target):
    """
    Simulates the GPU kernel checking if the hash is below the target.
    """
    # Mock hashing (SHA256 of the data)
    h = hashlib.sha256(share_data).digest()
    # Convert to integer (big-endian)
    hash_int = int.from_bytes(h, 'big')

    return hash_int < pool_target

# --- Scenario: Pool sends new difficulty ---
# The miner MUST store this and update the kernel parameter immediately
pool_difficulty = 38.0 
current_target = difficulty_to_target(pool_difficulty)

# Miner generates a share
share_data = b'miner_generated_solution_data'

# The check
is_valid = check_share(share_data, current_target)

How Senior Engineers Fix It

  • Implement a State Machine: Treat the Stratum connection as a state machine where mining.set_difficulty immediately updates a global atomic variable or a thread-safe state object.
  • GPU Synchronization: When the difficulty changes, the host code must:
    1. Stop dispatching new work (or flag the old work as stale).
    2. Update the Metal MTLBuffer or kernel arguments with the new target (not just the difficulty integer).
    3. Resume dispatch.
  • Response Validation: strictly parse the JSON-RPC response for mining.submit. A valid submission returns true. If it returns false or an error object, log the exact error string from the pool (e.g., “Low difficulty share”) rather than just incrementing a local “Rejected” counter.
  • Hex Target Calculation: Ensure the target is formatted correctly as a 32-byte hex string (little-endian or big-endian depending on the Stratum variant) when submitting, though usually the pool handles the verification, the miner must ensure the ntime and nonce are valid for the assigned window.

Why Juniors Miss It

  • Assuming “Sent” means “Accepted”: Juniors often debug the network transport layer (is the packet sending?) rather than the application logic (is the data valid?).
  • Hardcoded Constants: It is common to hardcode difficulty for testing (e.g., diff = 38) and forget to replace it with the variable received from the pool.
  • Ignoring VarDiff: New miners often implement the initial mining.subscribe and mining.authorize perfectly but neglect the mining.set_difficulty notification, assuming the initial difficulty is static.
  • Hex/Int Confusion: Confusing the difficulty (an integer or float like 38) with the target (a massive hex string like 00000000ffff0000...). The pool checks the target, not the difficulty number.