incremental replications causing disproportional usagage of redshift disk space

Follow

1

Hartmut Leps

• August 11, 2017 14:35

If any of you are experiencing the same issue, it makes sense to automatically perform a deep copy once in a while on incrementally updated tables. We have this scheduled weekly and the job is quite fast (runs in 6min). We were able to reduce the disk usage by 25% after not performing this for over a month.

If your in interested in the procedure please drop me a line hartmut.leps@blacklane.com. 

• Comment actions Permalink

  Official comment

  

  Matthias Korn

  • October 26, 2017 09:55

  Hi Hartmut,

  I had the same experience with redshift. Normally, a native vacuum call to redshift should be sufficient, but I experienced that this is not possible anymore when the cluster is more than 50% full.

  We are eager to see your script, please post it here.

  Best,
  Matthias

Please sign in to leave a comment.