i have had problems integrating apache spark using google storage. especially be... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

sandGorgon on March 1, 2017 | parent | context | favorite | on: Ask HN: Is S3 down?

i have had problems integrating apache spark using google storage. especially because s3 is directly supported in spark.

if you are api compatible with s3, could you make it easy /possible to work with google storage inside spark?

remember i may or may not run my spark on Dataproc.

bluedonuts on March 9, 2017 [–]

You can use the Google cloud storage connector (https://cloud.google.com/hadoop/google-cloud-storage-connect...) which works with hadoop (and therefore spark).

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact