node.js - Properly handling context.succeed()/context.fail() with AWS S3 service calls in AWS Lambda -


i've searched through posts here (i.e. how structure sequential aws service calls within lambda given calls asynchronous?) , elsewhere, , can't seem find 1 little bit of information me past annoying issue. when have lambda function iterates through loop, , within loop makes call s3.putobject(), runs short-circuit issue when trying deal context.succeed()/context.fail() or older context.done(null, 'msg') way of closing lambda process.

i.e. iteration needs call s3.putobject() current object uploaded, still output cloudwatch or possibly sqs/sns file successfully uploaded. however, of attempts @ putting type of closure function meets random results of getting file names, other times getting of file names, etc.

what best way it? i've attempted use q , async honest, i'm still learning of stuff..

below rough example of i'm attempting do:

function output(s3object){      s3.putobject(s3object, function(err, data){           if (err) {                console.log('there issue outputting object.', err);           } else {              // how close if have x number of incoming calls??           // context.done(null, 'success'); }   // , later in code calls output function // , note: should output of file names invocation uploads! (var = 0; < myrecords.length; a++){      output(myrecords[a]); }

but, said previously, attempts i've made far, mixed results.

successfully output object: mybucket/prefix/part_000000123432345.dat output object: mybucket/prefix/part_000000123432346.dat

but test of function outputs:

successfully output object: mybucket/prefix/part_000000123432346.dat

argh.

i'll give simple example using async can adapt:

var async = require('async');  var sleep = function(message, callback) {     settimeout(function() {         callback(null, "echo: " + message);     }, math.floor(math.random() * 2000)); };  exports.handler = function(event, context) {     async.map(['a', 'b', 'c', 'd', 'e'], sleep, context.done); };

here i've defined sleep() function takes message , callback, sleeps random amount of time between 0 , 2 seconds, echoes message callback.

we use async.map() invoke sleep() function asynchronously on 5 different messages. per the docs, callback function, in case context.done, called when all iterator functions have finished. run in lambda console , sure enough, get:

[   "echo: a",   "echo: b",   "echo: c",   "echo: d",   "echo: e" ]

so code might simple as:

async.map(myrecords, s3.putobject, context.done);

but have no way of testing that, leave part you.


Comments

Post a Comment

Popular posts from this blog

java - Spring Data JPA: Why findOne(id) executing delete query internally? -

i have 1 application using @manytomany annotation link 2 different entities 1 parent entity. not sure if have implemented properly. when execute .findone() on parent entity's repository find it, @ first fetch data along executing delete queries internally delete record mapping table (a table jpa creates after adding @manytomany annotation.). following generated sql queries , code snippet: java class @entity @table(name = "inventory") public class inventory { @id long id; // many other properties @manytomany(fetch=fetchtype.lazy) private list<stream> streams; @manytomany(fetch=fetchtype.lazy) private list<subject> subjects; } there no other annotation i've put in stream , subject entity class link inventory class. when run app, create following tables: inventory stream subject inventory_stream inventory_subject (inventory_stream , inventory_subject, 2 tables created automatically, don't k...
Read more

python - Mongodb How to add addtional information when aggregating? -

i beginner , wrote line pipeline works, wanna add other information output, screen name , or number of tweets.i tried add under $group gave me syntax error everytime here pipeline: def make_pipeline(): # complete aggregation pipeline pipeline = [ { '$match': { "user.statuses_count": {"$gt":99 }, "user.time_zone": "brasilia" } }, { "$group": { "_id": "$user.id", "followers": { "$max": "$user.followers_count" } } }, { "$sort": { "followers": -1 } }, { "$limit" : 1 } ]; i using on example : { "_id" : objectid("5304e2e3cc9e684aa98bef97"), "text" : "first week of school on :p", "in_reply_to_status_id" : null, "retweet_count" : null, ...
Read more

java - Incorrect order of records in M-M relationship in hibernate -

Image
i have 4 records in bridge table in mysql database. using @manytomany in hibernate, accurate order of employees incorrect incorrect of projects. following required material figure out doing mistake. this source. @entity @table(name="project") public class projectbean { @id @generatedvalue @column(name="project_id") private int projectid; @column(name="title") private string projecttitle; @manytomany(mappedby="projects") private collection<employeebean> employees; public projectbean() { employees = new arraylist<employeebean>(); } //getters & setters } @entity @table(name="employee") public class employeebean { @id @generatedvalue @column(name="employee_id") private int employee_id; @column(name="first_name") private string ...
Read more