Firestore & AWS Lambda integration

Reddit
Linkedin

The easy way...

If you ever tried to connect Firestore from aws lambda, you might find this problem, it was a pain in the ass.

Unable to import module ‘index’: Error at Function.Module._resolveFilename (module.js:469:15) at Function.Module._load (module.js:417:25) at Module.require (module.js:497:17) at require (internal/module.js:20:19) at Object.<anonymous> (/var/task/node_modules/firebase-admin/node_modules/grpc/src/node/src/grpc_extension.js:30:15) at Module._compile (module.js:570:32) at Object.Module._extensions..js (module.js:579:10) at Module.load (module.js:487:32) at tryModuleLoad (module.js:446:12)

The problem is that the Node.js SDK for Cloud Firestore uses gRPC to communicate, it is built on native code, and by default, AWS Lambda functions can’t run native code. If you install the firebase SDK and then compile your code.

It made it work, but I introduced a lot of stress, specially onboarding other developers, it was way too much overhead and I don’t need this kind of negativity in my life. I wanted to keep it simple and keep using the serverless framework in a single plug and play fashion as I was doing. So… gRPC again, I just needed a more powerful way to compile my projects, I did a lot of tests, a lot of reading and I was stressed… after a couple of days I came with this solution.

Serverless webpack plugin to the rescue

Install the serverless-webpack plugin

npm install --save-dev serverless-webpack

Then on your serverless.yml.

service: entr-auth

plugins:
  - serverless-webpack
  
custom:
   webpack:
    webpackConfig: './webpack.config.js' 
    packExternalModulesMaxBuffer: 1000000
    includeModules: true
    packagerOptions:
      scripts:
        - npm rebuild grpc --target=8.1.0 --target_arch=x64 --target_platform=linux --target_libc=glibc
        
provider:
  name: aws
  runtime: nodejs8.10
  
functions:
  handler: handler.hello

A small webpack config, you can add your own spicy.

const path = require('path')
const slsw = require('serverless-webpack')
const nodeExternals = require('webpack-node-externals')
const webpack = require('webpack')

module.exports = {
  entry: slsw.lib.entries,
  target: 'node',
  mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
  optimization: {
    minimize: false
  },
  performance: {
    hints: false
  },
  devtool: 'nosources-source-map',
  externals: [
    nodeExternals()
  ],
  plugins: [
    new webpack.IgnorePlugin(/^\/opt/)
  ],
  output: {
    libraryTarget: 'commonjs2',
    path: path.join(__dirname, '.webpack'),
    filename: '[name].js',
    sourceMapFilename: '[file].map'
  }
}

And that’s all! now you can easily use the firebase sdk as regular.

'use strict'
const admin = require('firebase-admin')
const serviceAccount = require('your_firebase_key')
admin.initializeApp({
   credential: admin.credential.cert(serviceAccount)
})
const firestore = admin.firestore()
const settings = { timestampsInSnapshots: true }

module.exports.hello = async (event) => {
   const data = firebase.collection('my_nice_collection')
    .get()
    .then(s => {
      // do whatever you need with your data
    })
}

Problem solves, now you can keep accessing firebase from the awesome serverless framework without all the overhead of using Docker and bla bla bla.

I will try my best to keep writing about all the struggles I find during my journey with serverless.

Enjoyed this post? Receive the next one in your inbox!

I hand pick all the best resources about Firebase and GCP around the web.


Not bullshit, not spam, just good content, promised 😘.


Reddit
Linkedin