Freegle Tech Report 18-08-31: Difference between revisions

From Freegle Wiki
Jump to navigationJump to search
m (→‎Direct.: <br>)
 
Line 25: Line 25:


'''Summary of  Client and Server Code Changes'''<br>
'''Summary of  Client and Server Code Changes'''<br>
Here are the more important ones which would affect members:
Here are the more important ones which would affect members:<br>
Add rating to mark as taken/received  
Add rating to mark as taken/received <br>
Add thumbs up and down for other freeglers
Add thumbs up and down for other freeglers<br>
Login with email/password sometimes fails to work
Login with email/password sometimes fails to work<br>
Add photos to user stories  
Add photos to user stories <br>
Move up/down buttons on chat  
Move up/down buttons on chat <br>
Chat ratings icons wrong way round  
Chat ratings icons wrong way round <br>
Ratings not showing on user profile popup  
Ratings not showing on user profile popup <br>
Put thumbs up/down ratings into buttons at bottom of chat on some screensizes to encourage use
Put thumbs up/down ratings into buttons at bottom of chat on some screensizes to encourage use<br>
Clicking multiple times on repost reminder mail can cause multiple reposts too soon.
Clicking multiple times on repost reminder mail can cause multiple reposts too soon.<br>
TAKENs not matching with OFFERs if messages arrive in a strange order  
TAKENs not matching with OFFERs if messages arrive in a strange order <br>


...and here are the ones which are minor, or only affect volunteers in ModTools, or are internal changes that you wouldn't notice:
...and here are the ones which are minor, or only affect volunteers in ModTools, or are internal changes that you wouldn't notice:<br>
Speed-up: Use bloodhound to improve typeahead function on post page and reduce server load
Speed-up: Use bloodhound to improve typeahead function on post page and reduce server load<br>
Retire function to disable chat review  
Retire function to disable chat review <br>
Speed-up: cache unread newsfeed count  
Speed-up: cache unread newsfeed count <br>
Make stories on the website match the ones we put in the newsletter
Make stories on the website match the ones we put in the newsletter<br>
Line breaks not showing in stories on ModTools
Line breaks not showing in stories on ModTools<br>
Show datestamp when there is a time gap for all types of chat message .
Show datestamp when there is a time gap for all types of chat message .<br>
Avoid ugly blank popup for availability if network or server is slow
Avoid ugly blank popup for availability if network or server is slow<br>
Don't ask when you're available if you've already been asked today to help members who are posting or replying multiple times.
Don't ask when you're available if you've already been asked today to help members who are posting or replying multiple times.<br>
Add option to never ask for the availability information again
Add option to never ask for the availability information again<br>
Don't keep showing the tooltip about the address button in chat  
Don't keep showing the tooltip about the address button in chat <br>
If you try to scroll in a chat just after opening it, it fights back. Stop that
If you try to scroll in a chat just after opening it, it fights back. Stop that<br>
Speedup: only fetch chat summary unless required
Speedup: only fetch chat summary unless required<br>
Change order of "missing" sections on ModTools dashboard  
Change order of "missing" sections on ModTools dashboard <br>
Slight code improvement which might help with duplicate chat messages  
Slight code improvement which might help with duplicate chat messages <br>
Improve help message for chat review  
Improve help message for chat review <br>
In ModTools, show email address of person who reported a spammer so that it's easy to let them know why it hasn't been approved Merge remote-tracking branch 'origin/master'
In ModTools, show email address of person who reported a spammer so that it's easy to let them know why it hasn't been approved Merge remote-tracking branch 'origin/master'<br>
Clarify text for new member approval
Clarify text for new member approval<br>
More work on predictions  
More work on predictions <br>
Remove obsolete event tracking code; we use inspectlet now, which is better  
Remove obsolete event tracking code; we use inspectlet now, which is better <br>
Add weekly alerts to prompt Freegle groups to move off Yahoo  
Add weekly alerts to prompt Freegle groups to move off Yahoo <br>
Newsfeed member notifications setting wrong way round  
Newsfeed member notifications setting wrong way round <br>
Clarify names of deleted/purged users
Clarify names of deleted/purged users<br>
Make PDF link a full URL to allow download from app  
Make PDF link a full URL to allow download from app <br>
Speed-up: cache typical reply time rather than calculate it on the fly each time
Speed-up: cache typical reply time rather than calculate it on the fly each time<br>
Speed-up: use exact matching for typeahead suggestions, which is quicker
Speed-up: use exact matching for typeahead suggestions, which is quicker<br>
Speed-up: don't bother suggesting subjects for TAKEN/RECEIVED messages, or messages from TN  
Speed-up: don't bother suggesting subjects for TAKEN/RECEIVED messages, or messages from TN <br>
Speed-up: check for spammer references when posting to newsfeed is slow. UT fixes.  
Speed-up: check for spammer references when posting to newsfeed is slow. UT fixes.  
Remove ability to disable chat review  
Remove ability to disable chat review <br>
Speed-up: better index unread newsfeed count
Speed-up: better index unread newsfeed count<br>
Speed-up: cache unread newsfeed count
Speed-up: cache unread newsfeed count<br>
Speed-up: count of unread newsfeed items is slow; cache it in the session  
Speed-up: count of unread newsfeed items is slow; cache it in the session <br>
Speed-up: exim log processing  
Speed-up: exim log processing <br>
Add extra case where we close and reopen the DB connection if we get an error  
Add extra case where we close and reopen the DB connection if we get an error <br>
Add and then fix thumbs up and down for other freeglers  
Add and then fix thumbs up and down for other freeglers <br>
Yahoo has broken the old style group URLs
Yahoo has broken the old style group URLs<br>
Messages should not expire at 90 days if autoreposts are set for longer than that
Messages should not expire at 90 days if autoreposts are set for longer than that<br>
Fix shortlinks broken by Yahoo changes
Fix shortlinks broken by Yahoo changes<br>
Messages which would trigger spam review not showing as unread in User2Mod chat
Messages which would trigger spam review not showing as unread in User2Mod chat<br>
Test of ML for no shows  
Test of ML for no shows <br>
Over 100 changes have been made for Travis implementation.
Over 100 changes have been made for Travis implementation.<br>
Using wrong GeoIP country database  
Using wrong GeoIP country database <br>
Log insert failing  
Log insert failing <br>
DST issue for times in digests  
DST issue for times in digests <br>
Mail mods when members post in the areas of groups they're an active mod on
Mail mods when members post in the areas of groups they're an active mod on<br>
Speed-up: use spatial indexing to find groups near a postcode more efficiently
Speed-up: use spatial indexing to find groups near a postcode more efficiently<br>
Update schema  
Update schema <br>
More experimentation with prediction
More experimentation with prediction<br>
Update lastaccess info for users who operate by email.
Update lastaccess info for users who operate by email.<br>
More OOF messages  
More OOF messages <br>
Duplicate tweets of messages
Duplicate tweets of messages<br>
Rename Profile to Predict
Rename Profile to Predict<br>
Improve bounce handling  
Improve bounce handling <br>
Speedup: slow query for Yahoo membership dump processing  
Speedup: slow query for Yahoo membership dump processing <br>
Coverage improvements  
Coverage improvements <br>
Speedup - chaseup messages putting a lot of load on DB server  
Speedup - chaseup messages putting a lot of load on DB server <br>
Clicking multiple times on repost reminder mail can cause multiple reposts too soon.
Clicking multiple times on repost reminder mail can cause multiple reposts too soon.<br>
More notification fixes  
More notification fixes <br>
More work on predictions - cache trained model  
More work on predictions - cache trained model <br>
Fix for push notifications, using wrong counts
Fix for push notifications, using wrong counts<br>
Email verification  
Email verification <br>
Many UT fixes.
Many UT fixes.<br>
Update status checking script for new monit version on Ubuntu 18.04.  
Update status checking script for new monit version on Ubuntu 18.04. <br>
More notification fixes for app
More notification fixes for app<br>
phpunit 7 fixes  
phpunit 7 fixes <br>
Remove obsolete events code - we use Inspectlet etc instead which are better  
Remove obsolete events code - we use Inspectlet etc instead which are better <br>
Update to PHP 7.2, Ubuntu 18.04, PHPUnit v7
Update to PHP 7.2, Ubuntu 18.04, PHPUnit v7<br>
More work on upgrading to FCM for notifications  
More work on upgrading to FCM for notifications <br>
Start migration to using Firebase for mobile app notifications
Start migration to using Firebase for mobile app notifications<br>
Add cron script to run predictions
Include chatcount and notifcount in push notifications for convenience of app <br>
Include chatcount and notifcount in push notifications for convenience of app  
Strip additional TN sig <br>
Strip additional TN sig  
Newsfeed digest subject line looks silly for introductions<br>
Newsfeed digest subject line looks silly for introductions
Much work on predictions<br>
Much work on predictions
More OOF messages <br>
Newsfeed digest subject line looks silly for introductions
Fix earlier merge issues<br>
More OOF messages  
Checking Facebook app notifications <br>
Fix earlier merge issues
Log slow API calls <br>
Checking Facebook app notifications  
More push notification error case handling<br>
Log slow API calls  
App notifications not working if FD Push Notifications setting is off.<br>
More push notification error case handling
Speed-up: introduce summary flag to enable us to get a basic list of chats more quickly <br>
App notifications not working if FD Push Notifications setting is off.
CLI script to merge groups<br>
Speed-up: introduce summary flag to enable us to get a basic list of chats more quickly  
Purge logs job running multiple times<br>
CLI script to merge groups
Many notification fixes <br>
Purge logs job running multiple times
Remove code which caches lists of chats. It's too risky, and we have a better way of improving performance. Coverage improvements - chat notification payload<br>
Many notification fixes  
Remove code which caches lists of chats. It's too risky, and we have a better way of improving performance. Coverage improvements - chat notification payload
  Speed-ups: minor DB reductions  
  Speed-ups: minor DB reductions  
Coveralls broken  
Coveralls broken <br>
UT fixes  
UT fixes <br>
Remove dummy test  
Remove dummy test <br>
Coveralls coverage tracking broken  
Coveralls coverage tracking broken <br>
Speedups: more reduction of DB ops for user objects and chatlists  
Speedups: more reduction of DB ops for user objects and chatlists <br>
Release updated schema
Release updated schema<br>
Speedup: reduce number of DB calls for user profiles  
Speedup: reduce number of DB calls for user profiles <br>


'''Questions & Answers'''
'''Questions & Answers'''<br>
Q. One member who is only on the one group has reported when searching messages they are getting results for locations which are out of our area.<br>
Q. One member who is only on the one group has reported when searching messages they are getting results for locations which are out of our area.<br>
A.It does show any within a defined area, working on straight line travel. It does not take into account difficulty.
A.It does show any within a defined area, working on straight line travel. It does not take into account difficulty.<br>


== Mobile App. ==
== Mobile App. ==

Latest revision as of 11:14, 3 October 2018

Freegle Tech Report August 2018 Messages 25816-25977 Members 113


General

Our website has slowed down a couple of times and one day posts failed to be sent out. All fixed once noticed. Edward has spent a lot of time this month, setting up a system which allows him to execute and test changes before implementing and also speeding up and looking at being able to predict members behaviours from their initial emails.

Direct.

Leading on from work to find out if we can predict those likely to collect promptly, it has been suggested that Short replies are bad, and also that a form of some type may help. Some mods do not like the idea of more forms, for which Edward has said he would look at having a quicker method for more experienced members.
Who's Travis? There are a shedload of code changes relating to Travis. A person? A band? Well, yes, but it's also something geekier and is important in that it can keep a blueprint of a website and test any proposed changes across the lot to see any repercussions of supposed fixes. This should cut down on bugs getting into the system.
FOP A couple of Moderators think it is in the wrong place and doubt the majority of people even notice it is there but are unsure where would be better. Any ideas? Edward says he is happy to change the wording if we can think of something else, but longer term this will probably be replaced by other function e.g. “must go before”, “can deliver”, “no rush”.
Predicting successful freegles (was Short replies are bad?)
Edward has done some number crunching, and much testing and has encouraging results in being able to predict from how people reply whether or not they are likely to be freeglers that people are happy with. There’s still more testing to do here. Much discussion on this - some don’t like us using technology to spy on members, some think that if it helps members have more success it’s worth doing..

NewsFeed

We ask freeglers to tell their stories, which appear in various places (newsfeed, mailed out, https://www.ilovefreegle.org/stories). Essex Council mentioned that it was hard to find photos of people freegling things, so I've added uploading a photo into those.

SMS alerts
The percentage, of members opting to use this has been fairly stable and once using it continue to do so. At the present rate it will cost us about £215 per month. It will be up to the board to decide if we are willing and able to pay this amount.

Summary of Client and Server Code Changes
Here are the more important ones which would affect members:
Add rating to mark as taken/received
Add thumbs up and down for other freeglers
Login with email/password sometimes fails to work
Add photos to user stories
Move up/down buttons on chat
Chat ratings icons wrong way round
Ratings not showing on user profile popup
Put thumbs up/down ratings into buttons at bottom of chat on some screensizes to encourage use
Clicking multiple times on repost reminder mail can cause multiple reposts too soon.
TAKENs not matching with OFFERs if messages arrive in a strange order

...and here are the ones which are minor, or only affect volunteers in ModTools, or are internal changes that you wouldn't notice:
Speed-up: Use bloodhound to improve typeahead function on post page and reduce server load
Retire function to disable chat review
Speed-up: cache unread newsfeed count
Make stories on the website match the ones we put in the newsletter
Line breaks not showing in stories on ModTools
Show datestamp when there is a time gap for all types of chat message .
Avoid ugly blank popup for availability if network or server is slow
Don't ask when you're available if you've already been asked today to help members who are posting or replying multiple times.
Add option to never ask for the availability information again
Don't keep showing the tooltip about the address button in chat
If you try to scroll in a chat just after opening it, it fights back. Stop that
Speedup: only fetch chat summary unless required
Change order of "missing" sections on ModTools dashboard
Slight code improvement which might help with duplicate chat messages
Improve help message for chat review
In ModTools, show email address of person who reported a spammer so that it's easy to let them know why it hasn't been approved Merge remote-tracking branch 'origin/master'
Clarify text for new member approval
More work on predictions
Remove obsolete event tracking code; we use inspectlet now, which is better
Add weekly alerts to prompt Freegle groups to move off Yahoo
Newsfeed member notifications setting wrong way round
Clarify names of deleted/purged users
Make PDF link a full URL to allow download from app
Speed-up: cache typical reply time rather than calculate it on the fly each time
Speed-up: use exact matching for typeahead suggestions, which is quicker
Speed-up: don't bother suggesting subjects for TAKEN/RECEIVED messages, or messages from TN
Speed-up: check for spammer references when posting to newsfeed is slow. UT fixes. Remove ability to disable chat review
Speed-up: better index unread newsfeed count
Speed-up: cache unread newsfeed count
Speed-up: count of unread newsfeed items is slow; cache it in the session
Speed-up: exim log processing
Add extra case where we close and reopen the DB connection if we get an error
Add and then fix thumbs up and down for other freeglers
Yahoo has broken the old style group URLs
Messages should not expire at 90 days if autoreposts are set for longer than that
Fix shortlinks broken by Yahoo changes
Messages which would trigger spam review not showing as unread in User2Mod chat
Test of ML for no shows
Over 100 changes have been made for Travis implementation.
Using wrong GeoIP country database
Log insert failing
DST issue for times in digests
Mail mods when members post in the areas of groups they're an active mod on
Speed-up: use spatial indexing to find groups near a postcode more efficiently
Update schema
More experimentation with prediction
Update lastaccess info for users who operate by email.
More OOF messages
Duplicate tweets of messages
Rename Profile to Predict
Improve bounce handling
Speedup: slow query for Yahoo membership dump processing
Coverage improvements
Speedup - chaseup messages putting a lot of load on DB server
Clicking multiple times on repost reminder mail can cause multiple reposts too soon.
More notification fixes
More work on predictions - cache trained model
Fix for push notifications, using wrong counts
Email verification
Many UT fixes.
Update status checking script for new monit version on Ubuntu 18.04.
More notification fixes for app
phpunit 7 fixes
Remove obsolete events code - we use Inspectlet etc instead which are better
Update to PHP 7.2, Ubuntu 18.04, PHPUnit v7
More work on upgrading to FCM for notifications
Start migration to using Firebase for mobile app notifications
Include chatcount and notifcount in push notifications for convenience of app
Strip additional TN sig
Newsfeed digest subject line looks silly for introductions
Much work on predictions
More OOF messages
Fix earlier merge issues
Checking Facebook app notifications
Log slow API calls
More push notification error case handling
App notifications not working if FD Push Notifications setting is off.
Speed-up: introduce summary flag to enable us to get a basic list of chats more quickly
CLI script to merge groups
Purge logs job running multiple times
Many notification fixes
Remove code which caches lists of chats. It's too risky, and we have a better way of improving performance. Coverage improvements - chat notification payload

Speed-ups: minor DB reductions 

Coveralls broken
UT fixes
Remove dummy test
Coveralls coverage tracking broken
Speedups: more reduction of DB ops for user objects and chatlists
Release updated schema
Speedup: reduce number of DB calls for user profiles

Questions & Answers
Q. One member who is only on the one group has reported when searching messages they are getting results for locations which are out of our area.
A.It does show any within a defined area, working on straight line travel. It does not take into account difficulty.

Mobile App.

I'm starting to roll out a new version 1.4.0 of the Android app.
As well as the usual catch up with the website code, we've switched to a new method of sending notifications ie Firebase Cloud Messaging (FCM) along with some improvements handling notifications.
This method will hopefully be better and allow us to remove notification tokens when the user uninstalls or re-installs the app.
The iOS app will hopefully follow soon.
For a while people using older versions of the app will continue to receive notifications the old way. However this technique is being retired so we will switch it off, so users need to upgrade to the new version of the app.
Chris
App hat


Mod Tools

Newsfeed Notifications. Some Moderators are unaware that to get notifications of Newsfeed posts made by their members, it needs to be switched on or off in their settings. ModTools->Settings->Personal
A moderator previously requested some changes to be made to reporting of spammers for Freegle groups only. These have now been done but are still on development.
Some time ago it was requested that images be retained for messages to Volunteers. This has now been done so you should no longer have to ask members to send them to your private email.
Q. Could you change the order of info displayed on the dashboard and swap Group Profile pics with Facebook Pages. It would make it easier to see the Facebook pages not linked info which happens more often these days.
A. Done but not live yet.

Tech Chat.

None for this month.

Trashnothing

Footers.

Some Moderators wish to add their own, others just to strip out Trash Nothings.
On the whole footers are stripped out of posts automatically. Some do get through for a while and small ones may escape detection.
You can get access to your group settings on Trashnothing and add or remove footers if you just ask Andrewn for the privs .


Yahoo.

September is the last month for groups remaining on Yahoo to be connected to Modtools and our own website. You need to let your members know in advance so you don’t lose them and put links on your front page. If you want help ask mentors@ilovefreegle.org At present there are around 50 groups which still need to move if they don’t wish to be disaffiliated. Some of them are out of contact, some have agreed we can take the non Yahoo members and they will just run a Yahoo group. It appears that groups who are to be disaffiliated will been given one month to change their mind before opening a new group covering their area.


Jean
August 2018
Freegle Tech Report



-
Back to Freegle Tech Report 18-07-31 Forward to Freegle Tech Report 18-09-30