Google Cloud StorageGoogle Cloud Storage allows world-wide storage and retrieval of any amount of data at any time. Show
Further information at the official Google Cloud Storage documentation website. This connector communicates to Cloud Storage via HTTP requests. Project Info: Alpakka Google Cloud StorageArtifactcom.lightbend.akka akka-stream-alpakka-google-cloud-storage 3.0.4 Snapshots are available JDK versionsAdopt OpenJDK 8 Adopt OpenJDK 11 Scala versions2.12.11, 2.13.3JPMS module nameakka.stream.alpakka.google.cloud.storageLicenseApache-2.0 Readiness levelCommunity-driven Since 1.1.0, 2019-07-03 Home pagehttps://doc.akka.io/docs/alpakka/currentAPI documentationAPI (Scaladoc) ForumsLightbend Discuss Release notesGitHub releasesIssuesGithub issuesSourceshttps://github.com/akka/alpakkaArtifactssbtval AkkaVersion = "2.6.14" val AkkaHttpVersion = "10.1.11" libraryDependencies ++= Seq( "com.lightbend.akka" %% "akka-stream-alpakka-google-cloud-storage" % "3.0.4", "com.typesafe.akka" %% "akka-stream" % AkkaVersion, "com.typesafe.akka" %% "akka-http" % AkkaHttpVersion, "com.typesafe.akka" %% "akka-http-spray-json" % AkkaHttpVersion )Maven<properties> <akka.version>2.6.14</akka.version> <akka.http.version>10.1.11</akka.http.version> <scala.binary.version>2.12</scala.binary.version> </properties> <dependencies> <dependency> <groupId>com.lightbend.akka</groupId> <artifactId>akka-stream-alpakka-google-cloud-storage_${scala.binary.version}</artifactId> <version>3.0.4</version> </dependency> <dependency> <groupId>com.typesafe.akka</groupId> <artifactId>akka-stream_${scala.binary.version}</artifactId> <version>${akka.version}</version> </dependency> <dependency> <groupId>com.typesafe.akka</groupId> <artifactId>akka-http_${scala.binary.version}</artifactId> <version>${akka.http.version}</version> </dependency> <dependency> <groupId>com.typesafe.akka</groupId> <artifactId>akka-http-spray-json_${scala.binary.version}</artifactId> <version>${akka.http.version}</version> </dependency> </dependencies>Gradledef versions = [ AkkaVersion: "2.6.14", AkkaHttpVersion: "10.1.11", ScalaBinary: "2.12" ] dependencies { implementation "com.lightbend.akka:akka-stream-alpakka-google-cloud-storage_${versions.ScalaBinary}:3.0.4" implementation "com.typesafe.akka:akka-stream_${versions.ScalaBinary}:${versions.AkkaVersion}" implementation "com.typesafe.akka:akka-http_${versions.ScalaBinary}:${versions.AkkaHttpVersion}" implementation "com.typesafe.akka:akka-http-spray-json_${versions.ScalaBinary}:${versions.AkkaHttpVersion}" }The table below shows direct dependencies of this module and the second tab shows all libraries it depends on transitively. Direct dependenciesOrganizationArtifactVersioncom.lightbend.akkaakka-stream-alpakka-google-common_2.123.0.4com.typesafe.akkaakka-http-spray-json_2.1210.1.11com.typesafe.akkaakka-http_2.1210.1.11com.typesafe.akkaakka-stream_2.122.6.14org.scala-langscala-library2.12.11Dependency treecom.lightbend.akka akka-stream-alpakka-google-common_2.12 3.0.4 com.github.jwt-scala jwt-spray-json_2.12 7.1.0 Apache-2.0 com.github.jwt-scala jwt-json-common_2.12 7.1.0 Apache-2.0 com.github.jwt-scala jwt-core_2.12 7.1.0 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 io.spray spray-json_2.12 1.3.6 Apache 2 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 com.google.auth google-auth-library-credentials 0.24.1 com.typesafe.akka akka-http-spray-json_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-http_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-http-core_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-parsing_2.12 10.1.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 io.spray spray-json_2.12 1.3.6 Apache 2 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 com.typesafe.akka akka-http_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-http-core_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-parsing_2.12 10.1.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 com.typesafe.akka akka-stream_2.12 2.6.14 Apache-2.0 com.typesafe.akka akka-actor_2.12 2.6.14 Apache-2.0 com.typesafe config 1.4.0 Apache-2.0 org.scala-lang.modules scala-java8-compat_2.12 0.8.0 BSD 3-clause org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 com.typesafe.akka akka-protobuf-v3_2.12 2.6.14 Apache-2.0 com.typesafe ssl-config-core_2.12 0.4.2 Apache-2.0 com.typesafe config 1.4.0 Apache-2.0 org.scala-lang.modules scala-parser-combinators_2.12 1.1.2 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.reactivestreams reactive-streams 1.0.3 CC0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 com.typesafe.akka akka-http-spray-json_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-http_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-http-core_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-parsing_2.12 10.1.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 io.spray spray-json_2.12 1.3.6 Apache 2 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 com.typesafe.akka akka-http_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-http-core_2.12 10.1.11 Apache-2.0 com.typesafe.akka akka-parsing_2.12 10.1.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 com.typesafe.akka akka-stream_2.12 2.6.14 Apache-2.0 com.typesafe.akka akka-actor_2.12 2.6.14 Apache-2.0 com.typesafe config 1.4.0 Apache-2.0 org.scala-lang.modules scala-java8-compat_2.12 0.8.0 BSD 3-clause org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 com.typesafe.akka akka-protobuf-v3_2.12 2.6.14 Apache-2.0 com.typesafe ssl-config-core_2.12 0.4.2 Apache-2.0 com.typesafe config 1.4.0 Apache-2.0 org.scala-lang.modules scala-parser-combinators_2.12 1.1.2 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.reactivestreams reactive-streams 1.0.3 CC0 org.scala-lang scala-library 2.12.11 Apache-2.0 org.scala-lang scala-library 2.12.11 Apache-2.0ConfigurationThe Storage connector shares its basic configuration with all the Google connectors in Alpakka. Additional Storage-specific configuration settings can be found in its own reference.conf. Store a file in Google Cloud StorageA file can be uploaded to Google Cloud Storage by creating a source of ByteStringByteString and running that with a sink created from GCStorage.resumableUploadGCStorage.resumableUpload. Scalacopysourceval sink = GCStorage.resumableUpload(bucketName, fileName, ContentTypes.`text/plain(UTF-8)`, chunkSize, metadata) val source = Source( List(ByteString(firstChunkContent), ByteString(secondChunkContent)) ) val result: Future[StorageObject] = source.runWith(sink) Javacopysourcefinal Sink<ByteString, CompletionStage<StorageObject>> sink = GCStorage.resumableUpload( bucketName(), fileName(), ContentTypes.TEXT_PLAIN_UTF8, chunkSize); final Source<ByteString, NotUsed> source = Source.from( Arrays.asList( ByteString.fromString(firstChunkContent), ByteString.fromString(secondChunkContent))); final CompletionStage<StorageObject> result = source.runWith(sink, system()); Download a file from Google Cloud StorageA source for downloading a file can be created by calling GCStorage.downloadGCStorage.download. It will emit an OptionOptional that will hold files data or will be empty if no such file can be found. If you need to download the specific version of the object in a bucket where object versioning is enabled, you can specify the generation. Scalacopysourceval downloadSource: Source[Option[Source[ByteString, NotUsed]], NotUsed] = GCStorage.download(bucketName, fileName) val downloadGenerationSource: Source[Option[Source[ByteString, NotUsed]], NotUsed] = GCStorage.download(bucketName, fileName, Some(generation)) Javacopysource final Source<Optional<Source<ByteString, NotUsed>>, NotUsed> downloadSource = GCStorage.download(bucketName(), fileName()); final Source<Optional<Source<ByteString, NotUsed>>, NotUsed> downloadGenerationSource = GCStorage.download(bucketName(), fileName(), generation()); Access object metadata without downloading object from Google Cloud StorageIf you do not need object itself, you can query for only object metadata using a source from GCStorage.getObjectGCStorage.getObject. If you need the specific version of the object metadata in a bucket where object versioning is enabled, you can specify the generation. Scalacopysourceval getObjectSource: Source[Option[StorageObject], NotUsed] = GCStorage.getObject(bucketName, fileName) val getObjectGenerationSource: Source[Option[StorageObject], NotUsed] = GCStorage.getObject(bucketName, fileName, Some(generation)) Javacopysource final Source<Optional<StorageObject>, NotUsed> getObjectSource = GCStorage.getObject(bucketName(), fileName()); final Source<Optional<StorageObject>, NotUsed> getObjectGenerationSource = GCStorage.getObject(bucketName(), fileName(), generation()); List bucket contentsTo get a list of all objects in a bucket, use GCStorage.listBucketGCStorage.listBucket. When run, this will give a stream of StorageObject. To get a list of both live and archived versions of all objects in a bucket where object versioning is enabled, the versions has to be set to true Scalacopysourceval listSource: Source[StorageObject, NotUsed] = GCStorage.listBucket(bucketName, Some(folder)) val listVersionsSource: Source[StorageObject, NotUsed] = GCStorage.listBucket(bucketName, Some(folder), versions) Javacopysource final Source<StorageObject, NotUsed> listSource = GCStorage.listBucket(bucketName(), folder); final Source<StorageObject, NotUsed> listVersionsSource = GCStorage.listBucket(bucketName(), folder, versions); Rewrite (multi part)Copy an Google Clouds Storage object from source bucket to target bucket using GCStorage.rewriteGCStorage.rewrite. When run, this will emit a single StorageObject with the information about the copied object. val result: Future[StorageObject] = GCStorage.rewrite(bucketName, fileName, rewriteBucketName, fileName).run Javacopysource final CompletionStage<StorageObject> result = GCStorage.rewrite(bucketName(), fileName(), rewriteBucketName, fileName()).run(system()); Apply Google Cloud Storage settings to a part of the streamIt is possible to make one part of the stream use different GoogleSettings from the rest of the graph. This can be useful, when one stream is used to copy files across regions with different service accounts. You can attach a custom GoogleSettings instance or a custom config path to a graph using attributes from GoogleAttributes: Scalacopysourceval newSettings = GoogleSettings(system).withProjectId("projectId") val listSource: Source[StorageObject, NotUsed] = GCStorage.listBucket(bucketName, None).withAttributes(GoogleAttributes.settings(newSettings)) Javacopysource final GoogleSettings newSettings = GoogleSettings.create(system()).withProjectId("projectId"); final Source<StorageObject, NotUsed> listSource = GCStorage.listBucket(bucketName()).withAttributes(GoogleAttributes.settings(newSettings)); Bucket managementBucket management API provides functionality for both Sources and Futures / CompletionStages. In case of the Future API user can specify attributes to the request in the method itself and as for Sources it can be done via method .withAttributes. For more information about attributes see: GCStorageAttributes and Attributes Make bucketIn order to create a bucket in Google Cloud Storage you need to specify its unique name. This value has to be set accordingly to the requirements. The bucket will be created in the given location. Scalacopysourceimplicit val sampleAttributes: Attributes = GoogleAttributes.settings(sampleSettings) val createBucketResponse: Future[Bucket] = GCStorage.createBucket(bucketName, location) val createBucketSourceResponse: Source[Bucket, NotUsed] = GCStorage.createBucketSource(bucketName, location) Javacopysource final Attributes sampleAttributes = GoogleAttributes.settings(sampleSettings); final CompletionStage<Bucket> createBucketResponse = GCStorage.createBucket(bucketName(), location, actorSystem, sampleAttributes); final Source<Bucket, NotUsed> createBucketSourceResponse = GCStorage.createBucketSource(bucketName(), location); Delete bucketTo delete a bucket you need to specify its name and the bucket needs to be empty. implicit val sampleAttributes: Attributes = GoogleAttributes.settings(sampleSettings) val deleteBucketResponse: Future[Done] = GCStorage.deleteBucket(bucketName) val deleteBucketSourceResponse: Source[Done, NotUsed] = GCStorage.deleteBucketSource(bucketName) Javacopysource final Attributes sampleAttributes = GoogleAttributes.settings(sampleSettings); final CompletionStage<Done> deleteBucketResponse = GCStorage.deleteBucket(this.bucketName(), actorSystem, sampleAttributes); final Source<Done, NotUsed> deleteBucketSourceResponse = GCStorage.deleteBucketSource(this.bucketName()); Get bucketTo get a bucket you need to specify its name. Scalacopysourceimplicit val sampleAttributes: Attributes = GoogleAttributes.settings(sampleSettings) val getBucketResponse: Future[Option[Bucket]] = GCStorage.getBucket(bucketName) val getBucketSourceResponse: Source[Option[Bucket], NotUsed] = GCStorage.getBucketSource(bucketName) Javacopysource final Attributes sampleAttributes = GoogleAttributes.settings(sampleSettings); final CompletionStage<Optional<Bucket>> getBucketResponse = GCStorage.getBucket(this.bucketName(), actorSystem, sampleAttributes); final Source<Optional<Bucket>, NotUsed> getBucketSourceResponse = GCStorage.getBucketSource(this.bucketName()); Running the example codeThe code in this guide is part of runnable tests of this project. You are welcome to edit the code and run it in sbt. Scalasbt > google-cloud-storage/test Javasbt > google-cloud-storage/test
|