screen-cast/src: Fix unsigned integer overflow

The fields of 'priv->video_format.max_framerate' are all of type uint32_t. Multiplying by G_USEC_PER_SEC can overflow, and equally, dividing a large numerical type by uint32_t can err too. Since the variable holding the result is int64_t, cast all uint32_t fields to int64_t before doing any maths on it. Spotted while trying to investigating an issue with framerates on HDMI screencasts. Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2762> (cherry picked from commit abfedcb0)

screen-cast/src: Fix unsigned integer overflow
d287e405 · Georges Basile Stavracas Neto · Robert Mader · b9931533 · d287e405
Commit d287e405 authored 2 years ago by Georges Basile Stavracas Neto Committed by Robert Mader 2 years ago
--- a/src/backends/meta-screen-cast-stream-src.c
+++ b/src/backends/meta-screen-cast-stream-src.c
@@ -624,8 +624,8 @@ meta_screen_cast_stream_src_maybe_record_frame (MetaScreenCastStreamSrc  *src,
      int64_t time_since_last_frame_us;

      min_interval_us =
-        ((G_USEC_PER_SEC * priv->video_format.max_framerate.denom) /
-         priv->video_format.max_framerate.num);
+        ((G_USEC_PER_SEC * ((int64_t) priv->video_format.max_framerate.denom)) /
+         ((int64_t) priv->video_format.max_framerate.num));

      time_since_last_frame_us = now_us - priv->last_frame_timestamp_us;
      if (time_since_last_frame_us < min_interval_us)