Hardening integer overflow guards in escape helper functions for atta…

[SABITHSAHEB]

server/util: Harden escape helpers against allocation size overflows

Summary

This patch adds integer overflow validation to several escape helper functions in server/util.c before calculating allocation sizes.

These functions derive buffer sizes from input lengths using expressions such as 3 * len + 1 and 10 * len. Without validation, extremely large values could cause arithmetic wraparound, resulting in an undersized allocation and potential memory corruption during subsequent writes.

Changes

• Added overflow checks before allocation size calculations.
• Prevented allocations when the computed size would overflow size_t.
• Preserved existing behavior for valid inputs.
• Added regression tests covering boundary and overflow cases.

Security Impact

This change hardens memory allocation logic against integer-overflow conditions that could otherwise lead to heap buffer overflows when processing unusually large inputs.

Testing

• Built successfully with GCC and MSVC.
• Added regression tests for safe boundary values and overflow conditions.
• Verified normal behavior remains unchanged for valid inputs.

Notes

• No API changes.
• No new dependencies.
• Backward compatible.

[notroj]

Using ap_assert in these cases is correct, don't remove that, if these paths are reachable you are just introducing some new error path which would crash differently anyway. Don't add new such paths anywhere, just use ap_assert throughout.

[SABITHSAHEB]

Makes sense, reverted to ap_assert across the board: restored the ones I'd removed and used it for the new checks too instead of the error-return paths. I also pulled the unrelated mod_core atoi changes and the cookie reverse-map sizing churn so this stays focused on the escape helpers. Pushed.