jason/qotnews
1
0
Fork 0
forked from tanner/qotnews
Code Issues Pull Requests Releases Wiki Activity

Commit Graph

  • 00954c6cac local browser scraper Jason 2020-11-11 09:26:54 +00:00
  • 637bc38476 fix mistake. Jason Schwarzenberger 2020-11-11 17:21:31 +13:00
  • 164b7e72c4 basically add declutter like capabilities. Jason Schwarzenberger 2020-11-11 17:16:04 +13:00
  • 3169af3002 hostname from settings. Jason Schwarzenberger 2020-11-11 09:46:27 +13:00
  • d588a60930 add source to searchable attributes. Jason Schwarzenberger 2020-11-11 09:37:54 +13:00
  • 408e2870b2 tzinfo and microdata schema urls. Jason Schwarzenberger 2020-11-10 16:51:27 +13:00
  • 44b8b36547 add data cast in query. Jason Schwarzenberger 2020-11-10 15:50:18 +13:00
  • 4f49684194 remove logos from utils.js Jason Schwarzenberger 2020-11-10 15:38:48 +13:00
  • 1d78b1c592 fix favicon url. Jason Schwarzenberger 2020-11-10 15:34:21 +13:00
  • 0374794536 Sitemap and Category to get favicon into icon property of story. Jason Schwarzenberger 2020-11-10 15:22:27 +13:00
  • 943a1cfa4f reader server Jason Schwarzenberger 2020-11-10 14:56:21 +13:00
  • 9cee370a25 tvnz icon Jason Schwarzenberger 2020-11-10 14:10:02 +13:00
  • 5efc6ef2d3 add related stories (in api only) Jason Schwarzenberger 2020-11-10 14:09:56 +13:00
  • 4ec50e20cb feed thread loop. Jason Schwarzenberger 2020-11-10 10:09:48 +13:00
  • c1b7877f4b remove limit. Jason Schwarzenberger 2020-11-09 17:54:50 +13:00
  • 7b8cbfc9b9 try to make feed only determined by the max age. Jason Schwarzenberger 2020-11-09 17:50:58 +13:00
  • bfa4108a8e Merge remote-tracking branch 'tanner/master' Jason Schwarzenberger 2020-11-09 16:08:28 +13:00
  • 0bd0d40a31 use json type in sqlite. Jason Schwarzenberger 2020-11-09 15:45:10 +13:00
  • 4e04595415 fix search. Jason Schwarzenberger 2020-11-09 15:44:44 +13:00
  • 006db2960c change to 3 days Jason 2020-11-09 01:36:51 +00:00
  • 1f063f0dac undo log level change Jason Schwarzenberger 2020-11-06 11:20:34 +13:00
  • 1658346aa9 fix news.py feed. Jason Schwarzenberger 2020-11-06 10:37:43 +13:00
  • 2dbc702b40 switch to python-dateutil for parser, reverse sort xml feeds. Jason Schwarzenberger 2020-11-06 10:02:39 +13:00
  • 1c4764e67d sort sitemap feed by lastmod time. Jason Schwarzenberger 2020-11-06 09:30:15 +13:00
  • ee49d2021e newsroom Jason 2020-11-05 20:28:55 +00:00
  • c391c50ab1 use localize Jason 2020-11-05 04:15:31 +00:00
  • 095f0d549a use replace. Jason Schwarzenberger 2020-11-05 16:57:08 +13:00
  • c21c71667e fix date issue. Jason Schwarzenberger 2020-11-05 16:41:15 +13:00
  • c3a2c91a11 update requirements.txt Jason Schwarzenberger 2020-11-05 16:33:50 +13:00
  • 0f39446a61 tz aware for use in settings. Jason Schwarzenberger 2020-11-05 16:30:55 +13:00
  • 351059aab1 fix excludes. Jason Schwarzenberger 2020-11-05 15:59:13 +13:00
  • 4488e2c292 add an excludes list of substrings for urls in the settings for sitemap/category. Jason Schwarzenberger 2020-11-05 15:51:59 +13:00
  • afda5b635c disqus test. Jason Schwarzenberger 2020-11-05 14:23:51 +13:00
  • 0fc1a44d2b fix issue in substack. Jason Schwarzenberger 2020-11-04 17:40:29 +13:00
  • 9fff1b9e46 avoid duplicate articles listed on the category page Jason Schwarzenberger 2020-11-04 17:14:42 +13:00
  • 16b59f6c67 try stop bad pages. Jason Schwarzenberger 2020-11-04 16:34:31 +13:00
  • 939f4775a7 better settings example. Jason Schwarzenberger 2020-11-04 15:52:34 +13:00
  • 9bfc6fc6fa scraper settings, ordering and loop. Jason Schwarzenberger 2020-11-04 15:47:12 +13:00
  • 6ea9844d00 remove useless try blocks. Jason Schwarzenberger 2020-11-04 15:37:19 +13:00
  • 1318259d3d imply referrer is substack. Jason Schwarzenberger 2020-11-04 15:21:07 +13:00
  • 98a0c2257c increase declutter timeout. Jason Schwarzenberger 2020-11-04 15:14:51 +13:00
  • e6976db25d fix tabs Jason Schwarzenberger 2020-11-04 15:04:20 +13:00
  • 9edc8b7cca move scraping for article content to files. Jason Schwarzenberger 2020-11-04 15:00:58 +13:00
  • 33e21e7f30 fix mistake. Jason Schwarzenberger 2020-11-04 12:45:01 +13:00
  • 892a99eca6 add + expander in place of collapser. Jason Schwarzenberger 2020-11-04 12:43:15 +13:00
  • d718d05a04 fix dates for newsroom. Jason Schwarzenberger 2020-11-04 11:53:16 +13:00
  • d1795eb1b8 add radionz and newsroom logos. Jason Schwarzenberger 2020-11-04 11:30:56 +13:00
  • 9f4ff4acf0 remove unnecessary sitemap.xml request. Jason Schwarzenberger 2020-11-04 11:22:15 +13:00
  • db6aad84ec fix mistake. Jason Schwarzenberger 2020-11-04 11:12:01 +13:00
  • 29f8a8b8cc add news site categories feed. Jason Schwarzenberger 2020-11-04 11:08:50 +13:00
  • 9a279d44b1 Add header to get content type tanner 2020-11-03 20:27:43 +00:00
  • abf8589e02 fix sitemap Jason 2020-11-03 10:53:40 +00:00
  • b759f46582 use extruct for opengraph/json-ld/microdata of articles Jason 2020-11-03 10:31:36 +00:00
  • 736cdc8576 fix mistake. Jason Schwarzenberger 2020-11-03 17:04:46 +13:00
  • 244d416f6e settings config of sitemap/substack publications. Jason Schwarzenberger 2020-11-03 17:01:29 +13:00
  • e506804666 Clean code up tanner 2020-11-03 03:45:56 +00:00
  • 5f98a2e76a Merge remote-tracking branch 'tanner/master' into master Jason Schwarzenberger 2020-11-03 16:44:02 +13:00
  • 0567cdfd9b move sort to render. Jason Schwarzenberger 2020-11-03 16:30:22 +13:00
  • 4f90671cec order feed by reverse chronological Jason Schwarzenberger 2020-11-03 16:21:23 +13:00
  • e63a1456a5 add logos. Jason Schwarzenberger 2020-11-03 16:07:07 +13:00
  • 76f1d57702 sitemap based feed. Jason Schwarzenberger 2020-11-03 16:00:03 +13:00
  • de80389ed0 add logos. Jason Schwarzenberger 2020-11-03 12:48:19 +13:00
  • 4e64cf682a add the bulletin. Jason Schwarzenberger 2020-11-03 12:41:16 +13:00
  • c5fe5d25a0 add substack.py top sites, replacing webworm.py Jason Schwarzenberger 2020-11-03 12:28:39 +13:00
  • 283a2b1545 fix webworm comments Jason 2020-11-02 22:06:43 +00:00
  • 0d6a86ace2 fix webworm dates. Jason Schwarzenberger 2020-11-03 10:31:14 +13:00
  • f23bf628e0 add webworm/substack as a feed. Jason Schwarzenberger 2020-11-02 16:07:05 +13:00
  • ca78a6d7a9 Move feed and Praw config to settings.py tanner 2020-11-02 02:26:54 +00:00
  • 7acce407e9 Fix index.html indentation tanner 2020-11-02 00:38:34 +00:00
  • 5281672000 Fix noscript font color tanner 2020-11-02 00:36:11 +00:00
  • e59acefda9 Remove Whoosh tanner 2020-11-02 00:22:40 +00:00
  • cbc802b7e9 Try Hackernews API twice tanner 2020-11-02 00:17:22 +00:00
  • 4579dfce00 Improve logging tanner 2020-11-02 00:13:43 +00:00
  • 0d16bec6f6 Fix table width CSS tanner 2020-11-01 00:47:18 +00:00
  • feba8b7aa0 Make qotnews work with WaPo tanner 2020-10-29 04:55:34 +00:00
  • ee5105743d Upgrade readability tanner 2020-10-29 01:24:13 +00:00
  • 72802a6fcf Show exerpt of hidden comments tanner 2020-10-27 00:41:36 +00:00
  • 99d3a234f4 Fix bug with rendering text nodes tanner 2020-10-10 21:07:54 +00:00
  • f95df227f1 Add instructions to download search server tanner 2020-10-04 21:21:19 +00:00
  • b82095ca7a Add buttons to collapse / expand comments tanner 2020-10-26 21:57:10 +00:00
  • 992c1c1233 Monkeypatch earlier tanner 2020-10-24 22:30:00 +00:00
  • 88d2216627 Add a script to delete a story tanner 2020-10-03 23:42:21 +00:00
  • 6cf2f01b08 Adjust feeds tanner 2020-10-03 23:41:57 +00:00
  • 607573dd44 Add buttons to convert <pre> to <p> tanner 2020-10-03 23:23:25 +00:00
  • c554ecd890 Add a line on UI to make search results obvious tanner 2020-08-14 03:58:11 +00:00
  • 6576eb1bac Adjust content-type request timeout tanner 2020-08-14 03:57:43 +00:00
  • 472af76d1a Adjust port tanner 2020-08-14 03:57:18 +00:00
  • 4727d34eb6 Delete displayed-attributes when init search tanner 2020-08-14 03:56:47 +00:00
  • 0e086b60b8 Remove business subreddit from feed tanner 2020-08-14 03:55:28 +00:00
  • b46ce36c63 Update requirements tanner 2020-07-08 05:24:32 +00:00
  • 9a449bf3ca Remove extra logging tanner 2020-07-07 20:55:13 +00:00
  • 0bd9f05250 Fix crash when HN feed fails tanner 2020-07-07 20:53:46 +00:00
  • 9c116bde4a Remove document img and ignore r/technology tanner 2020-07-06 21:44:16 +00:00
  • ebedaef00b Tune search rankings and attributes tanner 2020-07-06 21:43:57 +00:00
  • d7f0643bd7 Add more logging tanner 2020-07-04 22:39:38 +00:00
  • eb1137299d Remove article numbers tanner 2020-07-04 22:38:36 +00:00
  • 72d4a68929 Remove pre-fetching image tanner 2020-07-04 00:29:04 +00:00
  • f1c846acd0 Remove get first image tanner 2020-07-04 00:27:15 +00:00
  • 850b30e353 Add requests timeouts and temporary logging tanner 2020-07-04 00:25:41 +00:00
  • d614ad0743 Integrate with external MeiliSearch server tanner 2020-06-27 22:53:39 +00:00