Gyroflow: Video stabilization using gyroscope data

(github.com)

162 points | by nateb2022 3 days ago

11 comments

  • mikae1 23 hours ago
    I use this for near 100% of my video projects at work (with Sony cameras) and it's absolutely amazing. Sony is not exactly class leading when it comes to stabilization (like Panasonic). There's a Premiere and Resolve plugin-in these days.

    Fast guide:

    * Be sure to turn off any in camera stabilization in Sony cameras.

    * Be sure to take the added crop into account when composing.

    * The faster shutter speed, the better. Forget about 1/50 for 25p. There will be the most horrible artifacts. For 25p, use 1/100 or preferably use 1/200. For 50p, use 1/200 or preferably 1/400 etc.

    • JKCalhoun 20 hours ago
      > Forget about 1/50 for 25p. There will be the most horrible artifacts.

      (My ignorance,) is motion blur no longer a thing in modern digital?

      (I'm also confused: 1/50 is the shutter speed and 25p the frame rate?)

      • Gigachad 20 hours ago
        Normally you double the shutter speed compared to the frame rate to include a motion blur that looks good. But if you then apply stabilisation post processing, you end up with a shot that has motion blur yet isn’t moving. So you want to set a very fast shutter speed, and then you might introduce fake motion blur in post processing later that matches the movement of the stabilised video.
        • JKCalhoun 17 hours ago
          Interesting—and makes sense. (Perhaps motion blue becomes a post-process as well then, ha ha.)

          Kind of a tangent: One of the remarkable things to me about the "Dykstra flex" cameras developed for Star Wars was that the dolly/stepper-motors moved the camera while the shutter was open giving those fly-by shots full motion blur. Freeze any frame where there is a space battle and it is obvious.

          That small detail was not small at all in selling the effects of the film.

          But one of the effects guys joked that some team had borrowed the camera for some effects they were doing for a TV show or film—and they used Dykstra-flex in sort of a "stop motion" manner. He was dumbfounded why someone would move the camera, pause to expose a frame, move again to the next location, pause to expose. Just walking away, leaving motion-blur on the table…

          • dylan604 16 hours ago
            This is one of those things that pops up on fixed aperture cameras where the only way to control the exposure is with shutter speed. We used this when using GoPros in the early days of live action VR. We'd also run at 60fps. Any kind of motion blur would just become problematic when trying to stitch the footage together especially since the cameras were catching objects in different parts of the wide angle lenses.
            • Gigachad 10 hours ago
              You’ve got iso and ND filters still for exposure.
              • dylan604 9 hours ago
                ND filers slow down the shutter speed which will only increase the motion blur. When shooting in the sunny outdoors, you're going to be using the lowest ISO. The only thing left to control the exposure is the shutter speed. So with a fast shutter and 60fps also decreasing the exposure time the motion blur is going to be reduced as much as one can get.
                • Gigachad 8 hours ago
                  If you're on the lowest iso you could use an ND filter and bump the ISO a little higher giving some room to move up and down the ISO values to match the conditions while keeping the shutter speed fixed. That's kind of the primary function of an ND filter.

                  For video ideally you want both the shutter speed and aperture fixed since having the depth of field fluctuate in the video isn't ideal either.

                  • dylan604 8 hours ago
                    Sounds like someone that is used to running a single camera looking in a specific direction. Try running 6 cameras with one pointing in each direction so that every thing is seen. That's a small rig. Do the same thing 22 cameras running at the same time to make one image. You'll be just fine with allowing the camera to adjust the ISO for you.

                    I once thought similarly to you are thinking now. I quickly gave that up for the specific type of shooting discussed. You quickly realize that the software in post for stitching is really good at what it does.

      • _aavaa_ 20 hours ago
        Yes. It's confusing, which is why this is often discussed in terms of shutter angle, which makes this a litter easier to understand: https://www.youtube.com/watch?v=ehpdzt0JHUc
    • ipsum2 17 hours ago
      Gyroflow claims to support in body stabilization. I haven't done a side by side test to see if this works better.
  • hdersch 1 day ago
    I wonder how this compares to purely vision-based systems which use nothing but the images themselves for stabilization. Here are some quite old results of stabilization using image-based 3d-reconstruction of the scene which I wrote more than 10years ago, compared with other stabilization programs of that time (Deshaker, Adobe After Effects, Youtube). With todays improved hardware and progress in 3d-algorithms you may not need any additional gyroscopic data.

    https://www.youtube.com/watch?v=-m3fwhx3Z5g

    • _flux 1 day ago
      I think gyroscopic data still can have the edge if it has higher sampling rate than the video: then it could be used for removing blur from individual frames.

      I also expect purely gyroscopic approach to be much lighter compute-wise.

      • 7e 10 hours ago
        I expect you can recover most if not all of the gyroscopic data even from a blurred image, possibly at higher accuracy.
    • mikae1 14 hours ago
      In my experience, what you call vision-based systems don't even come close to Gyroflow. After I started using Gyroflow I haven't touched the Ronin gimbal many times.
      • 7e 10 hours ago
        A physical stabilizer like the Ronin is not what they mean. They mean vision based approaches which should be more accurate.
        • Gigachad 7 hours ago
          It doesn’t seem obvious that vision based stabilisation would be more accurate than a high accuracy gyroscope of the actual motion.

          At any rate if gyroflow obsoletes an actual gimbal it’s pretty close to perfect.

    • UltraSane 16 hours ago
      gyroscope data is much higher time resolution and is the ground truth vs guessing motion from pixel.
  • brad7 10 hours ago
    My friend built this which allows you to use Gyroflow natively within Final Cut, worth a look if that’s your NLE of choice: https://gyroflowtoolbox.fcp.cafe/
  • FrostKiwi 1 day ago
    So So useful <3 Stabilization = you have to zoom-in, loss of FOV. Depending on Action Cam and undistortion parameters, this can be different, sometimes too little, sometimes too much. Gyroflow allows you to dial it in. Lot's Stabilization in a particularly shaky spot and widest FOV everywhere else, smoothed between to be unnoticeable.
  • matsemann 1 day ago
    Haven't really used much other stabilization in post after modern gopros have gotten so good. Especially with the 360 variants (MAX and now MAX 2), it's buttery smooth (and infinite FOV means no cropping). Sometimes too smooth, I want to show how rough the cycling trail really was!
    • greatgib 1 day ago
      With very good daylight, Hypersmooth of Gopro is ok, but as soon as the conditions are a little bit less than ideal, watching the videos that we get out of the Gopro makes nauseous very fast.
  • rixed 1 day ago
    How is this different from usual sensor stabilisation techniques? Is it because it can adjust for a wider range of motion?
    • Gigachad 1 day ago
      Normal sensor stabilization only moves a tiny amount, It's more useful for photography for reducing micro jitters to get sharp photos. For video you need much more aggressive cropping and warping to undo the massive shake of walking with the camera.
  • po1nt 1 day ago
    As a drone pilot I used this before davinci studio implemented something similar. I highly recommend.
  • Gigachad 1 day ago
    I wanted to try this but sadly it seems my A7 iii doesn't record gyro data.
    • Zanfa 1 day ago
      There are external devices that can be attached to the camera to record gyro data, e.g https://docs.gyroflow.xyz/app/advanced-usage/using-external-... I just ordered one a few weeks ago and haven't received it yet, so can't talk about personal experience, but there's no technical reason it couldn't work well.
      • JosephRedfern 23 hours ago
        Out of curiosity - how do you sync gyro data with video timecode?
    • nnevatie 1 day ago
      My a7CR seems to be supported - will try this out later.
      • tecleandor 1 day ago
        Some months ago I tested footage of my A7C2 on Gyroflow expecting to do visual stabilization, and found my camera has a gyro (didn't know about it) and automatically adds that info to the videos.

        The result was pretty good and it was super easy to do it.

        • mikae1 23 hours ago
          Just be sure to turn off any in camera stabilization to get the best results.
          • Gigachad 8 hours ago
            In the guide videos they show you just have to sync it up by pressing the record button and end record button on both the gyroscope and the camera at the same time and this is apparently enough to calculate the time difference between the two clocks.
  • RicoElectrico 23 hours ago
    What's the deal with mdk-sdk? It is closed source, so there must be a very good reason to use it.

    For context, the webpage: https://www.qtav.org/blog/new-sdk.html

  • jauntywundrkind 1 day ago
    That this also helps so much with rolling shutter correction is wild! Epic.
  • nerderloo 1 day ago
    Such a fine piece of software.