210920 - TIL

210920 - TIL

# 한 것

## subreddit에서 가져온 데이터로 Reddit Reader 만들기

# 배운 것

##

The action attribute defines where the data gets sent. Its value must be a valid relative or absolute URL. If this attribute isn't provided, the data will be sent to the URL of the page containing the form — the current page.

The URL that processes the form submission.

In this example, the data is sent to an absolute URL — https://example.com:

Here, we use a relative URL — the data is sent to a different URL on the same origin:

※

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/form

※

https://developer.mozilla.org/en-US/docs/Learn/Forms/Sending_and_retrieving_form_data

## Get the data received in a Flask request

request.data: Contains the incoming request data as string in case it came with a mimetype Flask does not handle.

request.args: the key/value pairs in the URL query string

request.form: the key/value pairs in the body, from a HTML post form, or JavaScript request that isn't JSON encoded

request.files: the files in the body, which Flask keeps separate from form. HTML forms must use enctype=multipart/form-data or files will not be uploaded.

form. HTML forms must use enctype=multipart/form-data or files will not be uploaded. request.values: combined args and form, preferring args if keys overlap

args and form, preferring args if keys overlap request.json: parsed JSON data. The request must have the application/json content type, or use request.get_json(force=True) to ignore the content type. All of these are MultiDict instances (except for json). You can access values using: request.form['name']: use indexing if you know the key exists

request.form.get('name'): use get if the key might not exist

get if the key might not exist request.form.getlist('name'): use getlist if the key is sent multiple times and you want a list of values. get only returns the first value.

※

https://stackoverflow.com/questions/10434599/get-the-data-received-in-a-flask-request

## dictionary의 특정 key에 대한 value값을 기준으로 큰 순서대로 정렬하기

sort list by upvotes value

sorted(agg_list, key= lambda x: x["upvotes"], reverse=True)

※

https://docs.python.org/3/howto/sorting.html

## currency formatting (Flask, Jinja2)

{{post.upvotes|thousandFormat}} upvotes · r/{{post.subreddit}}

@app.template_filter() def thousandFormat(value): value = float(value) return "{:0,.0}".format(value)

result

3500 -> 3,500

※

https://stackoverflow.com/questions/55503515/flask-jinja-template-format-a-string-to-currency

✅ 작성한 코드

main.py

import requests from bs4 import BeautifulSoup from flask import Flask, render_template, request headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'} subreddits = [ "javascript", "reactjs", "reactnative", "programming", "css", "golang", "flutter", "rust", "django" ] app = Flask("DayEleven") @app.template_filter() def thousandFormat(value): #value = float(value) return "{:0,}".format(value) def get_subreddit_data(subreddit, agg_list): r_subreddit = requests.get(f"https://www.reddit.com/r/{subreddit}/top/?t=month", headers=headers) soup_subreddit = BeautifulSoup(r_subreddit.text, "html.parser") posts = soup_subreddit.find("div", {"class":"_31N0dvxfpsO6Ur5AKx4O5d"}).find_all("div", {"class":["_1oQyIsiPHYt6nx7VOmd1sz", "scrollerItem", "Post"]}) for post in posts: try: if "promoted" == post.find("span", {"class":"_2oEYZXchPfHwcf9mTMGMg8"}).get_text(): print("Promoted, get out of here!") continue except: pass title = post.find("h3", {"class":"_eYtD2XCVieq6emjKBH3m"}).get_text() try: link = post.find("a", {"class":"_3jOxDPIQ0KaOWpzvSQo-1s"})["href"] except: continue upvotes = post.find("div", {"class":"_1E9mcoVn4MYnuBQSVDt1gC"}).get_text() if 'k' in upvotes: k_idx = upvotes.index('k') upvotes = float(upvotes[:k_idx]) * 1000 post_dict = {"title":title, "link":link, "upvotes":int(upvotes), "subreddit":subreddit} agg_list.append(post_dict) return @app.route("/read") def read(): to_aggregate = request.args agg_list = [] for subreddit in to_aggregate: print(f"Scrapping r/{subreddit}...") get_subreddit_data(subreddit, agg_list) sorted_agg_list = sorted(agg_list, key= lambda x: x["upvotes"], reverse=True) return render_template("read.html", subreddit_to_aggregate=to_aggregate, aggregated_list=sorted_agg_list) @app.route("/") def home(): return render_template("home.html", subreddits=subreddits) app.run(host="0.0.0.0")

home.html

Reddit Reader Reddit Reader A service to aggregate all your favourite subreddits Select the subreddits you're interested on: {% for subreddit in subreddits %} {{subreddit}} {% endfor %} Aggregate

read.html

Reddit Reader Reddit Reader Reading: {% for subreddit in subreddit_to_aggregate %} r/{{subreddit}} {% endfor %} {% for post in aggregated_list %} {{post.title}} {{post.upvotes|thousandFormat}} upvotes · r/{{post.subreddit}} {% endfor %}

from http://refigo.tistory.com/27 by ccl(A) rewrite - 2021-08-21 10:00:47