Cara menggunakan python requests cookies

Question

larutan

Bagaimana cara menyimpan file cookies.txt gaya Netscape menggunakan Python 2.7 dan Permintaan? (How do I save a Netscape style cookies.txt file using Python 2.7 and Requests?)

Table of Contents Show

I am using requests to fetch a webpage and I would like to save the cookies in a Netscape‑style cookies.txt file. How can I achieve this? I have tried the following:

import requests
import cookielib

url = 'http://www.yahoo.com'
ua = ("Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) "
      "Gecko/20100101 Firefox/21.0")
headers = {'user‑agent': ua}

mcj = cookielib.MozillaCookieJar()
requests.get(url, headers=headers, cookies=mcj)
mcj.save('cookies.txt')
print open('cookies.txt').read()

but it just creates a cookie file with no cookies, so the output is:

# Netscape HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This is a generated file!  Do not edit.

larutan

(by Holy Mackerel)

larutan

How do I save a Netscape style cookies.txt file using Python 2.7 and Requests? (CC BY‑SA 3.0/4.0)

#python-2.7 #cookiejar #python-requests #cookielib

Bagian ini menunjukkan contoh program yang ditulis dengan Python yang menggambarkan bagaimana bekerja dengan Signature Versi 4 untuk Amazon Neptune. Contoh ini didasarkan pada contoh-contoh di bagian Proses Tanda Tangan Signature Versi 4 dalam Referensi Umum Amazon Web Services.

Untuk bekerja dengan program contoh ini, Anda memerlukan yang berikut ini:

Python 3.x diinstal pada komputer Anda, yang bisa didapatkan dari Situs Python. Program-program ini diuji menggunakan Python 3.6.
Pustakan permintaan Python, yang digunakan dalam skrip contoh untuk membuat permintaan web. Cara mudah untuk menginstal paket Python adalah dengan menggunakan pip, yang mendapat paket dari situs indeks paket Python. Anda kemudian dapat menginstal requests dengan menjalankan pip install requests pada baris perintah.
Kunci akses (access key ID dan secret access key) di variabel lingkungan bernama AWS_ACCESS_KEY_ID dan AWS_SECRET_ACCESS_KEY. Sebagai praktik terbaik, kami menyarankan Anda untuk tidak menanamkan kredensial dalam kode. Untuk informasi lebih lanjut, lihat Praktik Terbaik untuk Mengelola Access Keys AWS dalam Referensi Umum Amazon Web Services.
Wilayah klaster DB Neptune Anda dalam variabel lingkungan bernama SERVICE_REGION.
Jika Anda menggunakan kredensial sementara, Anda harus menentukan AWS_SESSION_TOKEN selain AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, dan SERVICE_REGION.

Contoh berikut menunjukkan cara membuat permintaan yang ditandatangani ke Neptune menggunakan Python. Permintaan tersebut membuat permintaan GET atau POST. Informasi autentikasi dilewatkan menggunakan header permintaan Authorization.

Contoh ini juga bekerja sebagai fungsi AWS Lambda. Untuk informasi selengkapnya, lihat Menyiapkan Lambda AWS untuk Autentikasi IAM Neptune.

Untuk membuat permintaan yang ditandatangani ke titik akhir Gremlin dan SPARQL Neptune

Buat file baru bernama neptunesigv4.py, lalu buka file tersebut dalam editor teks.

Salin kode berikut dan tempelkan ke file neptunesigv4.py.

# Amazon Neptune version 4 signing example (version v3)

# The following script requires python 3.6+
#    (sudo yum install python36 python36-virtualenv python36-pip)
# => the reason is that we're using urllib.parse() to manually encode URL
#    parameters: the problem here is that SIGV4 encoding requires whitespaces
#    to be encoded as %20 rather than not or using '+', as done by previous/
#    default versions of the library.


# See: https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
import sys, datetime, hashlib, hmac
import requests  # pip3 install requests
import urllib
import os
import json
from botocore.auth import SigV4Auth
from botocore.awsrequest import AWSRequest
from botocore.credentials import ReadOnlyCredentials
from types import SimpleNamespace
from argparse import RawTextHelpFormatter
from argparse import ArgumentParser

# Configuration. https is required.
protocol = 'https'

# The following lines enable debugging at httplib level (requests->urllib3->http.client)
# You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
#
# The only thing missing will be the response.body which is not logged.
#
# import logging
# from http.client import HTTPConnection
# HTTPConnection.debuglevel = 1
# logging.basicConfig()
# logging.getLogger().setLevel(logging.DEBUG)
# requests_log = logging.getLogger("requests.packages.urllib3")
# requests_log.setLevel(logging.DEBUG)
# requests_log.propagate = True


# Read AWS access key from env. variables. Best practice is NOT
# to embed credentials in code.
access_key = os.getenv('AWS_ACCESS_KEY_ID', '')
secret_key = os.getenv('AWS_SECRET_ACCESS_KEY', '')
region = os.getenv('SERVICE_REGION', '')

# AWS_SESSION_TOKEN is optional environment variable. Specify a session token only if you are using temporary
# security credentials.
session_token = os.getenv('AWS_SESSION_TOKEN', '')

### Note same script can be used for AWS Lambda (runtime = python3.6).
## Steps to use this python script for AWS Lambda
# 1. AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN and AWS_REGION variables are already part of Lambda's Execution environment
#    No need to set them up explicitly.
# 3. Create Lambda deployment package https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html
# 4. Create a Lambda function in the same VPC and assign an IAM role with neptune access

def lambda_handler(event, context):
    # sample_test_input = {
    #     "host": "END_POINT:8182",
    #     "method": "GET",
    #     "query_type": "gremlin",
    #     "query": "g.V().count()"
    # }

    # Lambda uses AWS_REGION instead of SERVICE_REGION
    global region
    region = os.getenv('AWS_REGION', '')

    host = event['host']
    method = event['method']
    query_type = event['query_type']
    query =  event['query']

    return make_signed_request(host, method, query_type, query)

def validate_input(method, query_type):
    # Supporting GET and POST for now:
    if (method != 'GET' and method != 'POST'):
        print('First parameter must be "GET" or "POST", but is "' + method + '".')
        sys.exit()

    # SPARQL UPDATE requires POST
    if (method == 'GET' and query_type == 'sparqlupdate'):
        print('SPARQL UPDATE is not supported in GET mode. Please choose POST.')
        sys.exit()

def get_canonical_uri_and_payload(query_type, query, method):
    # Set the stack and payload depending on query_type.
    if (query_type == 'sparql'):
        canonical_uri = '/sparql/'
        payload = {'query': query}

    elif (query_type == 'sparqlupdate'):
        canonical_uri = '/sparql/'
        payload = {'update': query}

    elif (query_type == 'gremlin'):
        canonical_uri = '/gremlin/'
        payload = {'gremlin': query}
        if (method == 'POST'):
            payload = json.dumps(payload)

    elif (query_type == 'openCypher'):
        canonical_uri = '/openCypher/'
        payload = {'query': query}

    elif (query_type == "loader"):
        canonical_uri = "/loader/"
        payload = json.loads(query)

    elif (query_type == "status"):
        canonical_uri = "/status/"
        payload = {}

    elif (query_type == "gremlin/status"):
        canonical_uri = "/gremlin/status/"
        payload = {}

    elif (query_type == "openCypher/status"):
        canonical_uri = "/openCypher/status/"
        payload = {}

    elif (query_type == "sparql/status"):
        canonical_uri = "/sparql/status/"
        payload = {}

    else:
        print(
            'Third parameter should be from ["gremlin", "sparql", "sparqlupdate", "loader", "status] but is "' + query_type + '".')
        sys.exit()
    ## return output as tuple
    return canonical_uri, payload

def make_signed_request(host, method, query_type, query):
    service = 'neptune-db'
    endpoint = protocol + '://' + host

    print()
    print('+++++ USER INPUT +++++')
    print('host = ' + host)
    print('method = ' + method)
    print('query_type = ' + query_type)
    print('query = ' + query)

    # validate input
    validate_input(method, query_type)

    # get canonical_uri and payload
    canonical_uri, payload = get_canonical_uri_and_payload(query_type, query, method)
    print('payload = {}'.format(payload))

    # assign payload to data or params
    data = payload if method == 'POST' else None
    params = payload if method == 'GET' else None

    # create request URL
    request_url = endpoint + canonical_uri

    # create and sign request
    creds = SimpleNamespace(
        access_key=access_key, secret_key=secret_key, token=session_token, region=region,
    )

    request = AWSRequest(method=method, url=request_url, data=data, params=params)
    SigV4Auth(creds, service, region).add_auth(request)

    r = None

    # ************* SEND THE REQUEST *************
    if (method == 'GET'):

        print('++++ BEGIN GET REQUEST +++++')
        print('Request URL = ' + request_url)
        r = requests.get(request_url, headers=request.headers, verify=False, params=params)

    elif (method == 'POST'):

        print('\n+++++ BEGIN POST REQUEST +++++')
        print('Request URL = ' + request_url)
        r = requests.post(request_url, headers=request.headers, verify=False, data=data)

    else:
        print('Request method is neither "GET" nor "POST", something is wrong here.')

    if r:
        print()
        print('+++++ RESPONSE +++++')
        print('Response code: %d\n' % r.status_code)
        response = r.text
        r.close()
        print(response)

        return response

help_msg = '''
    export AWS_ACCESS_KEY_ID=[MY_ACCESS_KEY_ID]
    export AWS_SECRET_ACCESS_KEY=[MY_SECRET_ACCESS_KEY]
    export AWS_SESSION_TOKEN=[MY_AWS_SESSION_TOKEN]
    export SERVICE_REGION=[us-east-1|us-east-2|us-west-2|eu-west-1]

    python version >=3.6 is required.

    Examples: For help
    python3 program_name.py -h

    Examples: Queries
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q status
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q sparql/status
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q sparql -d "SELECT ?s WHERE { ?s ?p ?o }"
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a POST -q sparql -d "SELECT ?s WHERE { ?s ?p ?o }"
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a POST -q sparqlupdate -d "INSERT DATA { <https://s> <https://p> <https://o> }"
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q gremlin/status
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q gremlin -d "g.V().count()"
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a POST -q gremlin -d "g.V().count()"
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q openCypher/status
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q openCypher -d "MATCH (n1) RETURN n1 LIMIT 1;"
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a POST -q openCypher -d "MATCH (n1) RETURN n1 LIMIT 1;"
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q loader -d '{"loadId": "68b28dcc-8e15-02b1-133d-9bd0557607e6"}'
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a GET -q loader -d '{}'
    python3 program_name.py -ho your-neptune-endpoint -p 8182 -a POST -q loader -d '{"source": "source", "format" : "csv", "failOnError": "fail_on_error", "iamRoleArn": "iam_role_arn", "region": "region"}'

    Environment variables must be defined as AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and SERVICE_REGION.
    You should also set AWS_SESSION_TOKEN environment variable if you are using temporary credentials (ex. IAM Role or EC2 Instance profile).

    Current Limitations:
    - Query mode "sparqlupdate" requires POST (as per the SPARQL 1.1 protocol)
            '''

def exit_and_print_help():
    print(help_msg)
    exit()

def parse_input_and_query_neptune():


    parser = ArgumentParser(description=help_msg, formatter_class=RawTextHelpFormatter)
    group_host = parser.add_mutually_exclusive_group()
    group_host.add_argument("-ho", "--host", type=str)
    group_port = parser.add_mutually_exclusive_group()
    group_port.add_argument("-p", "--port", type=int, help="port ex. 8182, default=8182", default=8182)
    group_action = parser.add_mutually_exclusive_group()
    group_action.add_argument("-a", "--action", type=str, help="http action, default = GET", default="GET")
    group_endpoint = parser.add_mutually_exclusive_group()
    group_endpoint.add_argument("-q", "--query_type", type=str, help="query_type, default = status ", default="status")
    group_data = parser.add_mutually_exclusive_group()
    group_data.add_argument("-d", "--data", type=str, help="data required for the http action", default="")

    args = parser.parse_args()
    print("parsed input: ")
    print(args)

    # Read command line parameters
    host = args.host
    port = args.port
    method = args.action
    query_type = args.query_type
    query = args.data

    if (access_key == ''):
        print('!!! ERROR: Your AWS_ACCESS_KEY_ID environment variable is undefined.')
        exit_and_print_help()

    if (secret_key == ''):
        print('!!! ERROR: Your AWS_SECRET_ACCESS_KEY environment variable is undefined.')
        exit_and_print_help()

    if (region == ''):
        print('!!! ERROR: Your SERVICE_REGION environment variable is undefined.')
        exit_and_print_help()

    if host is None:
        print('!!! ERROR: Neptune DNS is missing')
        exit_and_print_help()

    host = host + ":" + str(port)
    make_signed_request(host, method, query_type, query)


if __name__ == "__main__":
    print('here')
    parse_input_and_query_neptune()

Di terminal, arahkan ke lokasi file neptunesigv4.py.

Masukkan perintah berikut, ganti access key, kunci rahasia, dan Wilayah dengan nilai yang benar.

export AWS_ACCESS_KEY_ID=MY_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=MY_SECRET_ACCESS_KEY
export SERVICE_REGION=us-east-1 or us-east-2 or us-west-1 or us-west-2 or ca-central-1 or
                      sa-east-1 or eu-north-1 or eu-west-1 or eu-west-2 or eu-west-3 or eu-central-1 or me-south-1 or af-south-1 or
                      ap-east-1 or ap-northeast-1 or ap-northeast-2 or ap-southeast-1 or ap-southeast-2 or ap-south-1 or
                      cn-north-1 or cn-northwest-1 or
                      us-gov-east-1 or us-gov-west-1

Jika Anda menggunakan kredensial sementara, Anda harus menentukan AWS_SESSION_TOKEN selain AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, dan SERVICE_REGION.

export AWS_SESSION_TOKEN=MY_AWS_SESSION_TOKEN

Masukkan salah satu perintah berikut untuk mengirim permintaan yang ditandatangani ke instans DB Neptune. Contoh ini menggunakan Python versi 3.6.

Status Titik Akhir

python3.6 neptunesigv4.py -ho your-neptune-endpoint -a GET -q status

Gremlin

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a GET -q gremlin -d "g.V().count()"

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a POST -q gremlin -d "g.V().count()"

Status Gremlin

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a GET -q gremlin/status

SPARQL

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a GET -q sparql -d "SELECT ?s WHERE { ?s ?p ?o }"

PEMBARUAN SPARQL

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a POST -q sparql -d "SELECT ?s WHERE { ?s ?p ?o }" -p 8182 -a POST -q sparqlupdate -d "INSERT DATA { <https://s> <https://p> <https://o> }"

Status SPARQL

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a GET -q sparql/status

OpenCypher

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a GET -q openCypher -d "MATCH (n1) RETURN n1 LIMIT 1;"

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a POST -q openCypher -d "MATCH (n1) RETURN n1 LIMIT 1;"

Status OpenCypher

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a GET -q openCypher/status

Loader

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a GET -q loader -d '{"loadId": "68b28dcc-8e15-02b1-133d-9bd0557607e6"}'

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a GET -q loader -d '{}'

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p 8182 -a POST -q loader -d '{"source": "source", "format" : "csv", "failOnError": "fail_on_error", "iamRoleArn": "iam_role_arn", "region": "region"}'

Sintaks untuk menjalankan skrip Python adalah sebagai berikut:

python3.6 neptunesigv4.py -ho your-neptune-endpoint -p port -a GET|POST -q gremlin|sparql|sparqlupdate|loader|status -d "string0data"

SPARQL UPDATE memerlukan POST.

kode python Belajar Python Requests Requests Python Get cookie Python

Cara menggunakan python requests cookies

larutan

larutan

larutan

Pos Terkait

Periklanan

BERITA TERKINI

Toplist Popular

Periklanan

Terpopuler

Token Data

Periklanan

Tentang Kami

Legal

Dukungan

Social