Modeling and publishing a follower-based feed with Meteor -
i'm working on simple app user can follow other users. users can star posts. , user's feed composed of posts have been starred users follow. pretty simple actually. however, gets complicated in mongo , meteor...
there 2 way of modeling can think of:
a user has property,
following
, array of userids user follows. also, post has property,starrers
, array of userids have starred post. thing method publications relatively simple:meteor.publish 'feed', (limit) -> posts.find({starrers: {$in: meteor.users.findone(@userid).following}}, {sort: {date: -1}, limit:limit})
we aren't reactively listening user following, thats not bad now. main problem approach (1) individual documents become large , inefficient if 1000000 people star post. problem (2) pain keep track of information when user started following user or when user starred post.
the other way of doing having 2 more collections,
stars
,follows
. if user stars post, create document propertiesuserid
,postid
. if user follows user, create document propertiesuserid
,followid
. gives advantage of smaller document sizesusers
,posts
, complicated things when comes querying, because mongo doesn't handle joins!
now, did research , people seem agree second choice right way go. problem i'm having efficiently querying , publishing. based on discover meteor chapter advanced publications, created publication publishes posts starred user's followers -- sorted, , limited.
# helper handle stopping observechanges observer = (sub, func) -> handle = null sub.onstop -> handle?.stop?() () -> handle?.stop?() handle = func() meteor.publish 'feed', (limit) -> sub = userid = @userid followids = null eventids = null publishfollows = observer sub, () -> followids = {} follows.find({userid:userid}).observechanges added: (id, doc) -> followids[id] = doc.followid sub.added('follows', id, doc) publishstars() removed: (id) -> delete followids[id] sub.removed('follows', id) publishstars() publishstars = observer sub, () -> eventids = {} stars.find({userid: {$in: _.keys(followids)}).observechanges added: (id, doc) -> eventids[id] = null sub.added('stars', id, doc) publishevents() removed: (id) -> delete eventids[id] sub.removed('stars', id) publishevents() publishevents = observer sub, () -> events.find({_id: {$in: _.keys(eventids)}}, {sort: {name:1, date:-1}, limit:limit}).observechanges added: (id, doc) -> sub.added('events', id, doc) changed: (id, fields) -> sub.changed('events', id, fields) removed: (id) -> sub.removed('events', id)
while works, seems limited @ scale. particularly, have compile list of every starred post every follower. size of list grow quickly. huge $in
query against posts.
another annoyance querying feed on client after subscribe:
meteor.subscribe("feed", 20) posts = null tracker.autorun -> followers = _.pluck(follows.find({userid: meteor.userid()}).fetch(), "followid") starredpostids = _.pluck(stars.find({userid: {$in: followers}}).fetch(), "postid") posts = posts.find({_id: {$in: starredpostids}}, {sort: {date: -1}, limit: 20}).fetch()
its we're doing work twice. first work on server publish feed. need go through exact same logic again on client posts...
my question here matter of design on everything. how can efficiently design feed based on followers staring posts? collection / collection schemas should use? how should create appropriate publication? how can query feed on client?
so turns out mongo , "non-relational" databases aren't designed relational data. thus, there no solution here mongo. i've ended using neo4j, sql work fine well.
Comments
Post a Comment