Merge branch 'master' of https://github.com/shirosaidev/stocksight

Author: shirosaidev

README.md

@@ -6,18 +6,11 @@
 [![Donate PayPal](https://img.shields.io/badge/Donate%20%24-PayPal-brightgreen.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=CLF223XAS4W72)
 
 # stocksight
-Crowd-sourced stock market analyzer and stock predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis. How much do emotions on Twitter and news headlines affect a stock's price? Let's find out...
+Stock market analyzer and stock predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis. How much do emotions on Twitter and news headlines affect a stock's price? Let's find out...
 
 ## About
-stocksight is a crowd-sourced stock market analysis open source software that uses Elasticsearch to store Twitter and news headlines data for stocks. stocksight analyzes the emotions of what the author writes and does sentiment analysis on the text to determine how the author "feels" about a stock. stocksight makes an aggregated analysis of all collected data from all sources.
+stocksight is an open source stock market analysis software that uses Elasticsearch to store Twitter and news headlines data for stocks. stocksight analyzes the emotions of what the author writes and does sentiment analysis on the text to determine how the author "feels" about a stock.
 
-Each user running stocksight has a unique fingerprint: specific stocks they are following, news sites and twitter users they follow to find information for those stocks. This creates a unique sentiment analysis for each user, based on what data sources they are getting stocksight to search. Users can have the same stocks, but their data sources could vary significantly creating different sentiment analysis for the same stock. stocksight website will allow each user to see other sentiment analysis results from other stocksight user app results and a combined aggregated view of all.
-
-## Stocksight website
-https://stocksight.diskoverspace.com
-Currently in BETA. Free signup. Once you are registered and login, you will be given a token which you need to add to your stocksight config file to upload your stocksight results to the website. Edit config.py and add the line
-
-`stocksight_token = "<your auth token>"`
 
 ## Slack workspace
 Join the conversation, get support, etc on [stocksight Slack](https://join.slack.com/t/stocksightworkspace/shared_invite/enQtNzk1ODI0NjA3MTM4LTA3ZDA0YzllOGNiM2I5ZjAzYWM2MjNmMjI0OTRlY2ZjYTk1NmM5YmEwMmMwOTE2OTNiMGZlNzdjZmZkM2RjM2U).
@@ -50,8 +43,6 @@ $ cd stocksight
 Stocksight Kibana dashboard
 <img src="https://github.com/shirosaidev/stocksight/blob/master/docs/stocksight-dashboard-kibana.png?raw=true" alt="stocksight kibana dashboard" />
 
-Stocksight website
-<img src="https://github.com/shirosaidev/stocksight/blob/master/docs/stocksight_website_screenshot.png?raw=true" alt="stocksight website dashboard" />
 
 ## Install - Docker
 
@@ -96,9 +87,7 @@ https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens
 
 4) Add twitter consumer key/access token and secrets to config.py
 
-5) Edit config.py and modify NLTK tokens required/ignored and twitter feeds you want to mine. NLTK tokens required are keywords which must be in tweet before adding it to Elasticsearch and uploaded to StockSight website (whitelist). NLTK tokens ignored are keywords which if are found in tweet, it will not be added to Elasticsearch and uploaded to StockSight website (blacklist).
-
-These settings (tokens, feeds) and optional keywords for tweets, are what creates your unique fingerprint and helps with the crowd-sourcing of the stock symbol data mining and sentiment accuracy on StockSight website. The higher number of unique and valid data sources, the more accurate the sentiment analysis average should be.
+5) Edit config.py and modify NLTK tokens required/ignored and twitter feeds you want to mine. NLTK tokens required are keywords which must be in tweet before adding it to Elasticsearch (whitelist). NLTK tokens ignored are keywords which if are found in tweet, it will not be added to Elasticsearch (blacklist).
 
 ### Examples
 
@@ -108,16 +97,10 @@ Run sentiment.py to create 'stocksight' index in Elasticsearch and start mining
 $ python sentiment.py -s TSLA -k 'Elon Musk',Musk,Tesla,SpaceX --debug
 ```
 
-Start mining and analyzing Tweets using keywords and upload sentiment results to stocksight website (requires website auth token) using the stock symbol and stocksight website symbol TSLA. The website symbol is the "tag name" for the sentiment data being uploaded
-
-```sh
-$ python sentiment.py -s TSLA -k 'Elon Musk',Musk,Tesla,SpaceX -U --debug
-```
-
 Start mining and analyzing Tweets using keywords and the stock symbol TSLA and follow any url links in tweets and performing sentiment analysis on the link web page as well as the tweet
 
 ```sh
-$ python sentiment.py -s TSLA -k 'Elon Musk',Musk,Tesla,SpaceX -l -U --debug
+$ python sentiment.py -s TSLA -k 'Elon Musk',Musk,Tesla,SpaceX -l --debug
 ```
 
 Start mining and analyzing Tweets from feeds in config using cached user ids from file (if you change any of the twitter  feeds in the config file, you need to delete this file and recreate it without -f)
@@ -150,7 +133,7 @@ To import the saved exported visualizations/dashboard, go to Kibana, click on ma
 ```
 usage: sentiment.py [-h] [-i INDEX] [-d] -s SYMBOL [-k KEYWORDS] [-a] [-u URL]
                     [-f FILE] [-l] [-n] [--frequency FREQUENCY]
-                    [--followlinks] [-U] [-w] [--noelasticsearch]
+                    [--followlinks] [-w] [--noelasticsearch]
                     [--overridetokensreq TOKEN [TOKEN ...]]
                     [--overridetokensignore TOKEN [TOKEN ...]] [-v] [--debug]
                     [-q] [-V]
@@ -162,10 +145,7 @@ optional arguments:
   -d, --delindex        Delete existing Elasticsearch index first
   -s SYMBOL, --symbol SYMBOL
                         Stock symbol you are interesed in searching for,
-                        example: TSLA This is used as the symbol tag on
-                        stocksight website. Could also be set to a tag name
-                        like 'elonmusk' or 'elon' etc. Cannot contain spaces
-                        and more than 25 characters.
+                        example: TSLA
   -k KEYWORDS, --keywords KEYWORDS
                         Use keywords to search for in Tweets instead of feeds.
                         Separated by comma, case insensitive, spaces are ANDs
@@ -183,7 +163,6 @@ optional arguments:
                         (default: 120 sec)
   --followlinks         Follow links on news headlines and scrape relevant
                         text from landing page
-  -U, --upload          Upload sentiment to stocksight website (BETA)
   -w, --websentiment    Get sentiment results from text processing website
   --noelasticsearch     Don't connect or add new docs to Elasticsearch
   --overridetokensreq TOKEN [TOKEN ...]
@@ -196,9 +175,8 @@ optional arguments:
   --debug               Debug message output
   -q, --quiet           Run quiet with no message output
   -V, --version         Prints version and exits
-  ```
   
-  ```
+  
 usage: stockprice.py [-h] [-i INDEX] [-d] [-s SYMBOL] [-f FREQUENCY] [-v]
                      [--debug] [-q] [-V]
 

config.py.sample

@@ -2,10 +2,10 @@ elasticsearch_host = "localhost"
 elasticsearch_port = 9200
 elasticsearch_user = ""
 elasticsearch_password = ""
-consumer_key = "wA4nfiLzLZn8nfJUms7W44Lu9"
-consumer_secret = "R3oYihq08zkWf3qLu5auiO0BcNzegs7gpW47oWgJCQuBAKyLr4"
-access_token = "912155670882820096-jr9ipJ6FqyzcCc6v4c6ahPaQZmjx5JY"
-access_token_secret = "3GfwcUW4rsyVvPbKnkHfRW0iDZgzQcHbO9mtvYdGF0RRI"
+consumer_key = ""
+consumer_secret = ""
+access_token = ""
+access_token_secret = ""
 nltk_tokens_required = ("neuralink", "solar", "tesla", "@tesla", "#tesla", "tesla", "tsla", "#tsla", "elonmusk", "elon", "musk", "spacex", "starlink")
 nltk_min_tokens = 1
 nltk_tokens_ignored = ("win", "giveaway")
@@ -17,4 +17,4 @@ twitter_feeds = ["@elonmusk", "@cnbc", "@benzinga", "@stockwits",
                  "@muddywatersre", "@mcuban", "@AswathDamodaran", "@elerianm",
                  "@MorganStanley", "@ianbremmer", "@GoldmanSachs", "@Wu_Tang_Finance",
                  "@Schuldensuehner", "@NorthmanTrader", "@Frances_Coppola", "@bySamRo",
-                 "@BuzzFeed","@nytimes"]
+                 "@BuzzFeed","@nytimes"]

docs/_pages/home.md

@@ -8,7 +8,7 @@ header:
   cta_label: "<i class='fa fa-download' aria-hidden='true'></i> Download"
   cta_url: "https://github.com/shirosaidev/stocksight/releases/latest"
   caption:
-excerpt: "Crowd-sourced stock analyzer and stock predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis.<br /> <small><a href='https://github.com/shirosaidev/stocksight/releases/tag/v0.1-b.9'>Latest release v0.1-b.9</a></small><br /><br />
+excerpt: "Stock analyzer and stock predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis.<br /> <small><a href='https://github.com/shirosaidev/stocksight/releases/tag/v0.1-b.9'>Latest release v0.1-b.11</a></small><br /><br />
 {::nomarkdown}<iframe style='display: inline-block;' src='https://ghbtns.com/github-btn.html?user=shirosaidev&repo=stocksight&type=star&count=true&size=large' frameborder='0' scrolling='0' width='160px' height='30px'></iframe> <iframe style='display: inline-block;' src='https://ghbtns.com/github-btn.html?user=shirosaidev&repo=diskover&type=fork&count=true&size=large' frameborder='0' scrolling='0' width='158px' height='30px'></iframe>{:/nomarkdown}"
 github:
   - excerpt: '{::nomarkdown}<iframe style="display: inline-block;" src="https://ghbtns.com/github-btn.html?user=shirosaidev&repo=stocksight&type=star&count=true&size=large" frameborder="0" scrolling="0" width="160px" height="30px"></iframe> <iframe style="display: inline-block;" src="https://ghbtns.com/github-btn.html?user=shirosaidev&repo=stocksight&type=fork&count=true&size=large" frameborder="0" scrolling="0" width="158px" height="30px"></iframe>
@@ -19,5 +19,4 @@ intro:
 
 {% include feature_row id="intro" type="center" %}
 
-<p><strong>stocksight</strong> is a crowd-sourced stock analysis open source software that uses <strong>Elasticsearch</strong> to store Twitter and news headlines data for stocks. stocksight analyzes the emotions of what the author writes and does sentiment analysis on the text to determine how the author "feels" about a stock. stocksight makes an aggregated analysis of all collected data from all sources.</p>
-<p>Each user running stocksight has a unique fingerprint: specific stocks they are following, news sites and twitter users they follow to find information for those stocks. This creates a unique sentiment analysis for each user, based on what data sources they are getting stocksight to search. Users can have the same stocks, but their data sources could vary significantly creating different sentiment analysis for the same stock. stocksight website will allow each user to see other sentiment analysis results from other stocksight user app results and a combined aggregated view of all.</p>
+<p><strong>stocksight</strong> is an open source stock analysis software that uses <strong>Elasticsearch</strong> to store Twitter and news headlines data for stocks. stocksight analyzes the emotions of what the author writes and does sentiment analysis on the text to determine how the author "feels" about a stock.</p>