update notes

notes.org

@@ -3,10 +3,11 @@
 * Work
 ** Reoccuring Tasks
 *** TODO Catch-Up (Emails/Slack)
-DEADLINE: <2019-09-17 Tue .+1d>
+DEADLINE: <2019-09-27 Fri .+1d>
 :PROPERTIES:
-:LAST_REPEAT: [2019-09-16 Mon 08:50]
+:LAST_REPEAT: [2019-09-26 Thu 08:48]
 :END:
+- State "DONE"       from "WAIT"       [2019-09-26 Thu 08:48]
 - State "DONE"       from "TODO"       [2019-09-16 Mon 08:50]
 :PROPERTIES:
  :LAST_REPEAT: [2019-09-02 Mon 23:08]
@@ -19,24 +20,24 @@ CLOCK: [2019-09-16 Mon 08:19]
 CLOCK: [2019-08-29 Thu 08:25]--[2019-08-29 Thu 08:55] =>  0:30
 CLOCK: [2019-08-28 Wed 08:59]--[2019-08-28 Wed 09:05] =>  0:06
 :END:
-DEADLINE: <2019-09-17 Tue .+1d>
+DEADLINE: <2019-09-27 Fri .+1d>
 *** TODO Code Reviews
-DEADLINE: <2019-09-03 Tue .+1d>
+DEADLINE: <2019-09-27 Fri .+1d>
 :PROPERTIES:
-:LAST_REPEAT: [2019-09-02 Mon 23:08]
+:LAST_REPEAT: [2019-09-26 Thu 08:49]
 :END:
+- State "DONE"       from "WAIT"       [2019-09-26 Thu 08:49]
 - State "DONE"       from "PROJ"       [2019-09-02 Mon 23:08]
 :LOGBOOK:
 CLOCK: [2019-08-30 Fri 08:13]--[2019-08-30 Friday 8:25] =>  0:12
 :END:
-**** TODO [[https://github.com/BrandwatchLtd/frontend/pull/10180][AFA-28]]
 ** Sprint Work
 *** Sprint Ceremonies
 :LOGBOOK:
 CLOCK: [2019-08-27 Tue 14:00]--[2019-08-27 Tue 15:49] =>  1:49
 :END:
-*** TODO separate randomly failing tests
+*** DONE invalid startDate
-*** TODO invalid startDate
+CLOSED: [2019-09-26 Thu 08:49]
 [[https://logs.live.brandwatch.net/app/kibana#/doc/fb79e240-f24e-11e8-9bc6-79a720df60f0/hay0:dc-applog-2019.09.06/doc?id=rlwiB20BU5sr491KBDH1&_g=(refreshInterval:(pause:!t,value:0),time:(from:'2019-09-06T00:00:00.000Z',mode:absolute,to:'2019-09-06T23:11:00.000Z'))][log link]]
 
 Theory: project timezone is something luxon does not know. Projecttimezone in
@@ -53,6 +54,34 @@ fallback.
 
 The good thing is that most likely the user will not see a huge difference by
 looking at the mentions. So it's better to show utc instead of failing for sure.
+*** DONE investigate lost trainings
+    CLOSED: [2019-10-07 Mon 10:02]
+
+"Goose Island Brand Values", project: Amy, client: Staff - Integration
+
+userId 212083879
+projectId 1998259083
+queryIds 1999839102 1999916473
+classifierId categoryId 6754396
+subCategoryIds 6754397 6754398 6754399
+
+First observation, summary call
+https://app.brandwatch.com/fe-api/projects/1998259083/classifiers/trainings?_=1569391751072
+
+has trainingSet always as ~null~
+
+Turns out that we don't hande when fetching a classifier or categroy fails.
+
+If fetching a classifier fails, the interface actully looks like all training
+data is lost - BUT IT's actually NOT.
+
+But still not sure if a network issue is the cause of this.
+
+**** Trying what happens when I stall the requests made
+- [X] /projects/162524241/queries/summary
+- [ ] /projects/162524241/classifiers/trainings/8885308
+- [ ] /projects/162524241/categories
+- [ ] /projects/162524241/data/mentions
 
 *** ARCHIVE                                                       :ARCHIVE:
 :PROPERTIES:
@@ -291,6 +320,59 @@ CLOSED: [2019-08-29 Thu 08:51]
 :LOGBOOK:
 CLOCK: [2019-08-27 Tue 16:00]--[2019-08-27 Tue 17:16] =>  1:16
 :END:
+*** DONE Catchup with Sammy
+CLOSED: [2019-09-17 Tue 07:54]
+**** Iris Bug
+**** mixpanel tracking
+**** hide cats in rules and followup bugs
+**** unreproducable training data is lost STS-890
+**** startDate endDate null when getting mentions
+**** LD/module PR needs update - BE is in place
+*** DONE Answer Erik
+CLOSED: [2019-09-17 Tue 12:52]
+
+#+BEGIN_QUOTE
+When a user is trying to add a non-Owned Facebook page, and enters a URL for
+that page, we need to make a call (to the token service?) to get any available
+user token and use that to get page contents. Does this API endpoint already
+exist, or do we need to create one? Do we need to be worried about rate
+limiting? If there are no available user tokens, do we want an error or
+redirect to the auth token flow? (Mitch/Claudio that last one might be for
+you)
+#+END_QUOTE
+
+With "...to get the page contents" you mean to get the Facebook Page name, Icon,
+Description etc, correct? That's at least what we do in Analytics so far to be
+able to let the user verify that he picked the right one.
+
+To get the facebook page data you can also directly used the Facebook Graph API
+using the current short lived user token you currently have (assuming that
+checking if the user is already authenticated with Facebook OR authenticating
+with Facebook is done before that step)
+
+As I'm proposing to talk the Graph API instead and using the user token I don't
+think rate limits will be an issue. There is no such thing of rate limits in the
+token service nor in BW API (afaik).
+
+Generally speaking, any (long lived) user or page token we have in the backend,
+we should never hand out to the client as it somewhat breaks they proposed
+_security_ model Facebook proposes.
+
+
+- The endpoints to get a list of authenticated pages, the # of remaining
+  hashtags and storing hashtags for a authenticated page do already exist in BW API.
+~/instagramHashtags/facebookPages?clientId=${clientId}~
+** fb token service
+*** TODO investigate access_token null
+INVITE SECRET ~iPh5foo2ief4uv,i~ mit line break
+
+https://app.brandwatch.com/fbauth/3f509c305307099e1b58d4a8ce84510b
+https://app.brandwatch.com/fbauth/3f509c305307099e1b58d4a8ce84510b
+*** DONE fix log level in stackoverflow
+CLOSED: [2019-09-24 Tue 18:50]
+
+Deployed to live. The root cause was that Stackdriver does not evaluate ~level~
+at all for ~severtity~. You need to set that manually as well /o\.
 ** Howtos
 *** setup mitmproxy
 1. write a script to modify the response
@@ -309,6 +391,22 @@ CLOCK: [2019-08-27 Tue 16:00]--[2019-08-27 Tue 17:16] =>  1:16
 
 2. ~mitmproxy --mode reverse:https://bwjsonapi.stage.brandwatch.net -p 9999 -s <scriptName>~
 3. ~./runInDevelopment --apiUrl=http://localhost:9999~
+*** mitmproxy scripts
+**** throttle all request in random order
+#+BEGIN_SRC python
+from threading import Timer
+from random import random
+from mitmproxy import http
+
+def request(flow):
+    resumeLater = Timer(random() * 2, flow.resume)
+    flow.intercept()
+    resumeLater.start();
+#+END_SRC
+*** jq usage
+Get ids from a json response ~{ results: [{id: 4}, ...] }~
+
+~curl URL | jq '.results[] | .id'~
 
 ** Things to pass on
 *** TODO kitchen duty calendar
@@ -322,7 +420,41 @@ CLOCK: [2019-09-02 Mon 14:00]--[2019-09-02 Mon 16:00] =>  2:00
 :END:
 *** DONE prepare for BrightView Onboarding
 CLOSED: [2019-09-10 Tue 21:29] DEADLINE: <2019-09-03 Tue> SCHEDULED: <2019-08-29 Thu>
+*** DONE [[org:org-mode.org][org-mode lightning talk]]
+CLOSED: [2019-09-26 Thu 08:50]
+*** TODO separate randomly failing unit tests
+There are some randomly failing unit tests in the frontend.
+We currently retry ALL backbone frontend unit tests when they fail.
+
+This slows down CI and does not really help in identifying these.
+
+As a intermedate solution we want to separate these randomly failing unit tests
+in an own directory to be able to separate them from unit tests that work fine.
+
+This enables us to get rid of retrying all of them when some test fails.
+
+**** Failing tests
+
+- [X] [[https://jenkins.brandwatch.net/blue/organizations/jenkins/BrandwatchLtd%2Ffrontend/detail/master/2413/pipeline][DashboardToolbarView]]
+- [X] [[https://jenkins.brandwatch.net/blue/organizations/jenkins/BrandwatchLtd%2Ffrontend/detail/master/2369/pipeline][DashboardView]]
+- [X] [[https://jenkins.brandwatch.net/blue/organizations/jenkins/BrandwatchLtd%2Ffrontend/detail/PR-10357/1/pipeline/105][DataDownloadCollectionView]]
+- [X] [[https://jenkins.brandwatch.net/blue/organizations/jenkins/BrandwatchLtd%2Ffrontend/detail/PR-10357/2/pipeline/113][DataDownloadFormView]]
+- [X] [[https://jenkins.brandwatch.net/blue/organizations/jenkins/BrandwatchLtd%2Ffrontend/detail/PR-10357/1/pipeline][InsightsCentral_wrapDashboardView]]
+- [X] [[https://jenkins.brandwatch.net/blue/organizations/jenkins/BrandwatchLtd%2Ffrontend/detail/PR-10357/3/pipeline/100#step-126-log-20][GuiderView]]
+- [X] [[https://jenkins.brandwatch.net/blue/organizations/jenkins/BrandwatchLtd%2Ffrontend/detail/PR-10357/6/pipeline/96][InisightsCentral FilterContextMenuView]]
+- [X] QueryBuilderWriteView
+  : Expected: false
+  : Received: "The test \"QueryBuilderWriteView rendering auto fetch of preview
+  :           results on language change does not trigger fetching when query
+  :           is new or was not validated (validatedSettings are empty)\"
+  :           added a new child element to body, please remove it:
+  : <div class=\"validation-tip notification-error rounded smallpadding-vertical singlepadding-horizontal\"></div>"
+
+  Solved most of them by stubbing jquery.showValidation in various places.
+-
+** Axiom WG
 * Private
 
-** [[org:~/org/steuererklaerung.org][Steuererklärung 2018]]
+** DONE [[org:~/org/steuererklaerung.org][Steuererklärung 2018]]
+CLOSED: [2019-09-26 Thu 08:50]
 ** [[org:ausbau/][Ausbau]]