Skip to content

Conversation

@mobias17
Copy link

Overview

This PR addresses deserialization issues #477 #476 #405 with schema models in both REST and WebSocket APIs. The changes ensure that Pydantic's model_validate() method properly handles discriminated unions by delegating to custom from_dict() logic.

Problem Statement

The codebase uses oneOf schemas extensively for filter types (SymbolFilters, ExchangeFilters, AssetFilters) and user data stream events (ExecutionReport, BalanceUpdate, etc.). These models have custom deserialization logic in their from_dict() methods that handles discriminator-based type resolution.

However, several code paths use model_validate() instead of from_dict():

  1. REST API :

Models do not evaluate to a one_of_model and takte the model_validate branch

parsed = json.loads(response.text)
is_list = isinstance(parsed, list)
is_oneof = is_one_of_model(response_model)
is_flat_list = is_list and (
len(parsed) == 0 or
(not isinstance(parsed[0], list) if is_list else False)
)
if (is_list and not is_flat_list) or not response_model:
data_function = lambda: parsed
elif is_oneof or is_list or hasattr(response_model, "from_dict"):
data_function = lambda: response_model.from_dict(parsed)
else:
data_function = lambda: response_model.model_validate(parsed)

  1. WebSocket API :
    same as under the Rest Api

is_oneof = self.is_one_of_model(response_model)
if is_oneof or hasattr(response_model, "from_dict"):
data_function = lambda: response_model.from_dict(ws_response)
elif response_model:
data_function = lambda: response_model.model_validate(ws_response)
else:
data_function = lambda: ws_response

  1. User Data Streams (utils.py#L775):

for callback in callbacks:
if response_model:
if isinstance(payload, list):
parsed = [parse_user_event(item, response_model) for item in payload]
else:
parsed = parse_user_event(payload, response_model)
callback(parsed)
else:
callback(payload)

parse_user_event()

try:
instance = model_cls.model_validate(payload)
validator_field_map = get_validator_field_map(response_model_cls)
kwargs = {"actual_instance": instance}
if validator_field := validator_field_map.get(model_cls.__name__):
kwargs[validator_field] = instance
return response_model_cls(**kwargs)

When Pydantic's native model_validate() is called on models, it bypasses the custom discriminator logic in from_dict(), resulting in actual_instance=None and failed deserialization.

Root Cause

Pydantic v2's model_validate() and from_dict() take different routes:

  • from_dict(): Executes custom logic → discriminator resolution → proper type instantiation
  • model_validate(): Uses Pydantic's native validation → bypasses custom discriminator logic → fails for oneOf
  • **data unpacking: Constructs model directly → no validation or custom logic

The issue occurs because the response handling code calls model_validate(parsed_dict), but the models need from_dict(parsed_dict) to work correctly. The Models are not recognised as oneOf models.

Solution

1. Override model_validate() in Models

Add a model_validate() class method that intercepts dictionary inputs and delegates to from_dict():

Files Modified:

Implementation:

@classmethod
def model_validate(cls, obj: Any) -> Self:
    """Validate and deserialize using custom from_dict logic."""
    # If obj is a dict, use from_dict to handle oneOf deserialization properly
    if isinstance(obj, dict):
        return cls.from_dict(obj)
    # Otherwise use Pydantic's default validation
    return super().model_validate(obj)

2. Override model_validate() in Parent Response Models

Since ExchangeInfoResponse contains nested oneOf filter models, it also needs the override to ensure nested deserialization works correctly:

Files Modified:

Implementation:

@classmethod
def model_validate(cls, obj: Any) -> Self:
    """Validate and deserialize using custom from_dict logic."""
    # If obj is a dict, use from_dict to handle nested oneOf deserialization properly
    if isinstance(obj, dict):
        return cls.from_dict(obj)
    # Otherwise use Pydantic's default validation
    return super().model_validate(obj)

Important: The from_dict() method must call super().model_validate() (not cls.model_validate()) to avoid infinite recursion:

@classmethod
def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
    if not isinstance(obj, dict):
        return super(ExchangeInfoResponse, cls).model_validate(obj)
    
    _obj = super(ExchangeInfoResponse, cls).model_validate({...})
    return _obj

Impact

Fixed Code Paths

  1. REST API Exchange Info: Filter deserialization now works correctly for both from_dict() and model_validate() calls
  2. WebSocket API Exchange Info: Nested filter models in WebSocket responses deserialize properly
  3. User Data Streams: Events like ExecutionReport, BalanceUpdate, etc. now resolve to correct types in receive_loop()

Test Coverage

All test cases now pass:

  • ✅ REST API ExchangeInfoResponse with filters (from_dict)
  • ✅ REST API ExchangeInfoResponse with filters (model_validate)
  • ✅ WebSocket API ExchangeInfoResponse with filters (from_dict)
  • ✅ WebSocket API ExchangeInfoResponse with filters (model_validate)
  • ✅ User Data Stream event deserialization via parse_user_event()

Why This Approach?

This solution maintains backward compatibility while fixing the core issue:

  1. Non-breaking: from_dict() still works exactly as before
  2. Transparent: model_validate() now "just works" for models
  3. Consistent: Both code paths (from_dict and model_validate) now produce identical results
  4. Future-proof: Any new code using model_validate() will automatically get correct behavior

Alternative Implementation

The alternative to overwriting the model_validate() for those models could be to ensure that they are recognised by is_one_of() so that for websocket and rest api it would take the from_dict(). However this would not work on the UserDataStream events in parse_user_event() as there is not OneOf Validation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant