Use icu4x for UTF-8 float(); remove an alloc by joshuamegnauth54 · Pull Request #7998 · RustPython/RustPython

Walkthrough

Integrates icu_decimal for UTF‑8 decimal parsing in float conversion, adds SliceFmtWriter for bounded formatting into a byte buffer, and removes the old char_to_decimal lookup and export.

Changes

ICU Decimal Parsing Integration

Sequence Diagram(s)

sequenceDiagram
  participant FloatStr as float_from_string
  participant Decimal as icu_decimal::Decimal
  participant Writer as SliceFmtWriter
  participant FloatParse as std::parse
  FloatStr->>Decimal: try_from_str(trimmed_utf8)
  alt parse succeeds
    Decimal-->>FloatStr: decimal object
    FloatStr->>Writer: write_str(formatted)
    Writer-->>FloatStr: formatted buffer
    FloatStr->>FloatParse: parse buffer
  else parse fails
    Decimal-->>FloatStr: error
    FloatStr->>FloatParse: parse original bytes
  end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Suggested reviewers

• youknowone
• ShaharNaveh

Poem

🐰 I nibble digits snug and neat,
icu parses each tiny beat,
SliceFmtWriter stacks them tight,
Old tables hop away from sight,
Floats now leap on airy feet ✨

❌ Failed checks (1 warning)

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

• X
• Mastodon
• Reddit
• LinkedIn

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

crates/vm/src/utils.rs (1)

115-117: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Fix source slice bounds in partial copy logic.

On Line 116, &s.as_bytes()[to_copy..] is the wrong half of the input and can trigger copy_from_slice length mismatch panics when truncating. Use the prefix slice instead.

Suggested fix

-        self.buf[self.written..self.written + to_copy].copy_from_slice(&s.as_bytes()[to_copy..]);
+        self.buf[self.written..self.written + to_copy].copy_from_slice(&s.as_bytes()[..to_copy]);

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/vm/src/utils.rs` around lines 115 - 117, The partial-copy currently
takes the wrong suffix slice (&s.as_bytes()[to_copy..]) causing length
mismatches; change the source to the prefix slice of length to_copy (use
&s.as_bytes()[..to_copy]) when calling copy_from_slice so the copied length
equals to_copy, and keep updating self.written as before in the function that
computes to_copy via self.remainder().

crates/vm/src/utils.rs (1)

117-127: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Source slice index is inverted — will panic on every call.

Line 120 uses &s.as_bytes()[to_copy..] which yields the bytes after to_copy, with length s.len() - to_copy. The destination slice has length to_copy. This mismatch causes copy_from_slice to panic in nearly all cases (e.g., s = "hello", to_copy = 5 → source is empty, destination is 5 bytes).

The slice should be [..to_copy] (the first to_copy bytes).

Proposed fix

 impl fmt::Write for SliceFmtWriter<'_> {
     fn write_str(&mut self, s: &str) -> fmt::Result {
         let to_copy = self.remainder().min(s.len());
-        self.buf[self.written..self.written + to_copy].copy_from_slice(&s.as_bytes()[to_copy..]);
+        self.buf[self.written..self.written + to_copy].copy_from_slice(&s.as_bytes()[..to_copy]);
         self.written += to_copy;
         if to_copy == s.len() {
             Ok(())
         } else {
             Err(fmt::Error)
         }
     }
 }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/vm/src/utils.rs` around lines 117 - 127, The write_str implementation
in impl fmt::Write for SliceFmtWriter<'_> copies the wrong source slice (using
&s.as_bytes()[to_copy..]) which produces a length mismatch and panics; change
the source slice to the first to_copy bytes (&s.as_bytes()[..to_copy]) so it
matches the destination length, keeping the remainder() call and updates to
self.buf and self.written intact in the write_str method.