Republishing Implementation

From Freegle Wiki

Jump to: navigation, search

OBSOLETE PAGE

This page has been superseded. Please go to Freegle Direct - Introduction 


There are some earlier thoughts at Freegle Republishing Application.  
These need reviewing to spot anything else we should do.

Contents

Domains

This is driven via republish.freegle.in, largely because that was easy to get the DNS set up for. The domain is separate in case we need to hive this off to another host.

Receiving mail

The republisher receives mail by republisher@republisher.freegle.in being an email-only member of the group.

Mails are processed to extract:

  • The sender
  • The date
  • The message id
  • The subject

Database tables

There are the following tables.

  • groups lists the groups on which the republisher is active.
    • groupname is the short name of the group, e.g. EdinburghFreegle.
    • grouppublish is a flag to indicate whether this group is publishing.
    • groupdaystoshow controls how many days of posts the republisher will display.
    • userpublishdefault is a flag which controls whether posts are published without explicit user permission. This is normally 0.
    • facebookpage is the URL of a Facebook page for this group, e.g. http://www.facebook.com/pages/EdinburghFreegle/151105274929488.
    • facebookpageid is the unique ID of the Facebook page
    • facebookpagename is a friendly name for the Facebook page, displayed on menus
    • facebookfooter is included as a footer on messages sent from Facebook
    • groups_latlng contains the locations of groups, imported as a view from the Freegle database.
    • key is used to authenticate session changes
    • changerequest is a pending change of settings which is waiting for a user to confirm
    • userquerytext is the template for the mail sent to users to confirm/refuse republication. It can contain some substitution strings:
      • $group is expanded to the short group name.
      • $okurl is the URL sent to a user which confirms republication.
      • $notokurl is the URL sent to a user which confirms republication.
      • $republishurl is the URL of the RSS feed for the group.
      • $facebookpage' is expanded to the facebookpage value.
    • attachments stores (at the moment) the first attachment detected in incoming messages, where that attachment is stored on the Yahoo Group.
    • offerkeyword is the preferred OFFER keyword.  Similarly for takenkeyword, wantedkeyword, receivedkeyword.
    • repostoffer and repostwanted are the number of days before the Post Again link appears.
    • pointlinksat determines where links in RSS point at.
      • 0 means Yahoo
      • 1 means Facebook
  • users lists information about known users.
    • useremail is the email of the user, from which messages have been received.
    • userpublish is a flag indicating whether this user is currently publishing subject lines. 0 = no, 1 = anyone, 2 = facebook.
    • publishcontent is a similar flag indicating whether the user allows message content to be visible.
    • allowreply is a similar flag for replies.
    • lastqueryepoch is the seconds-since-epoch when this user was last sent a mail asking them to confirm/refuse republishing.
    • lastquerycreds is a random number used to make this link secure against hacking, so you can't change someone else's settings.
    • usergroup is the group that this setting is for (users might have different settings on different groups).
    • lastresponseepoch is the seconds-since-epoch when this user last confirmed/refused republishing.
  • messages contains information about messages which have been received.
    • epochtime is the seconds-since-epoch when this message was received.
    • date is a readable version of this.
    • id is the message id on the group.
    • groupname is the short name of the group the message is from.
    • subject is the subject of the message, excluding any group tag between square brackets.
    • from is the email address of the user who sent the message.
    • messagevisible controls whether the message is visible via republishing
    • messagetype is the type of the message:
      • 0 means unknown
      • 1 means OFFER
      • 2 means TAKEN
      • 3 means WANTED
      • 4 means RECEIVED
      • 5 means ADMIN
      • 6 means "File - "
    • messageavailable indicates if the OFFER/WANTED is still outstanding
    • relatedmessage is the id of the TAKEN for an OFFER or RECEIVED for a WANTED.
  • facebook is used to record information for users of the Facebook application
    • facebookid is the unique ID of the user as known by Facebook
    • facebookname is the Facebook friendly name of the user
    • email is the email the user specified for use with Yahoo (which might be different from the email address they use for Facebook)
    • verified is a flag indicating whether we have confirmed the email via a challenge
    • datechallenged is when we sent a challenge by email
    • dateverified is when the user clicked to respond (either positively or negatively)
    • key is the random key used in the challenge to make it secure
    • showdaysold is the user's preference for how many days of old posts to show
    • lastaccess is the date/time the user last accessed the Facebook application, but updated no more frequently than every 24 hours.
    • lastgroup is the last group the user selected when posting a message
    • lastlocation is the last location the user specified when posting a message
  • facebook_joining is used to maintain a pool of pre-approved memberships to speed up joining on Facebook.
    • groupid is the unique id of the group.
    • groupname is the Yahoo group name.
    • email is the made-up email address.
    • dateapplied is when we mailed the -subscribe address for this email.
    • dateapproved is when this email got approved onto the group.

Possibly the groups table should be combined with the main table for groups - but this would cause problems if we ever had to move to another host for this processing (which we might, if it's expensive).

At the moment we add entries to groups manually. It would be nicer to expose this via Perch, possibly to mods rather than geeks.

Flow

Joining

  1. User finds a Facebook page for a group.
  2. Clicks on the Join tab.
  3. Clicks on the Apply button.
  4. We try to grab an email address from the facebook_joining table; if we fail, give an error
  5. We update our tables for this email, facebook id and group.
  6. We top up facebook_joining if need be.

Incoming Mail

  1. Mail is received
  2. Check if group is republishing
  3. Check if the user has specified a republishing setting within the last 90 days
  4. If not, send them a challenge, no more frequently than once a day
  5. Either way, record the message, but set messagevisible appropriately

User Preferences

  1. Challenge mail is received
  2. User clicks on link
  3. Link verified via lastquerycreds
  4. userpublish updated and messagevisible updated for all emails for that user
  5. Extra republishing options presented; they can submit the form to update these.

There is currently no way for a user to request a new confirmation mail, though they can reuse the links in the first one.

We should also have a nag to get members to specify their preferences, in case they don't respond to the original mail.

RSS Feed

  1. Extract group name from URL
  2. Query messages table for visible messages for this group.

Group Settings

These can be accessed via the Edit button on the Facebook Republisher. This form records the change request in the DB, and verifies it via a mail to the -owner address which they have to click on/

Database purging

Not implemented yet.

Security

There are two risks:

  • User preferences are hacked. This is protected via lastquerycreds.
  • Mail is received which is not really from a group and gets republished anyway. We do some checking of the message to verify it comes from Yahoo, but there is an exposure here.

Facebook Applications

There are two types, IFRAME or FBML. Briefly:

  • FBML apps let you build them quickly in a way that matches Facebook's styles using the FBML format, and you can do (modified) Javascript using FBJS. There are some restrictions on what you can do (for example, submission of files in a form isn't possible and you have to do it inside an IFRAME).
  • IFRAME apps let you do anything but you have to put more effort into styles because they only support a very minimal set of FBML tags.

I've opted for FBML for now, for development speed - but I can imagine this changing later if someone who could do pretty got involved. Facebook are giving out mixed messages - they suggest they're moving towards IFRAME apps, but enhancing the FBML support.

Retrieved from "http://wiki.ilovefreegle.org/Republishing_Implementation"
Personal tools