Tutorial: Set Up PyCharm Professional with aDevelopment EndpointThis tutorial shows you how to connect the Python IDE running on your local machine to a developmentendpoint so that you can interactively run, debug, and test AWS Glue ETL (extract,transfer, andload) scripts before deploying them.To connect to a development endpoint interactively, you must have PyCharm Professionalinstalled. You can't do this using the free edition.The tutorial assumes that you have already taken the steps outlined in.Connecting PyCharm Professional to aDevelopment Endpoint.Create a new pure-Python project in PyCharm named legislators.Create a file named getpersonschema.py in the project withthe following content.In the Settings dialog box, expand the Build, Execution,Deployment section. Choose the Deploymentsubsection.Choose the + icon at the top of themiddle pane to add a new server. Give it a name and set its Typeto SFTP.Set the SFTP host to the Publicaddress of your development endpoint, as listed on its detailspage (choose the name of your development endpoint in the AWS Glue console todisplay the details page).Set the User name to glue.Set the Auth type to Key pair (OpenSSH or Putty).Set the Private key file by browsing to the location where yourdevelopment endpoint's private key file is located. Note that PyCharm only supportsDSA, RSA and ECDSA OpenSSH key types, and does not accept keys in Putty's privateformat.You can use an up-to-date version of ssh-keygen to generate a key-pairtype that PyCharm accepts, using syntax like the following.Note items 2 and 3 in the instructions in this screen. The script file that you createddoesimport pydevd. But in the call to settrace, it replaceslocalhost with 169.254.76.0, which is a special linklocal IP address that is accessible to your development endpoint.Choose Apply to save this default configuration.Choose the + icon at the top of the screen to create a new configurationbased on the default that you just saved.
Working with Consoles PyCharm enables you to use interactive consoles, thus making it possible to stay within the IDE, without the necessity to switch to the shell. Running Console.
In the drop-down menu, choose PythonRemote Debug. Name this configuration demoDevEndpoint,and choose OK.On the Run menu, choose Debug'demoDevEndpoint'. Your screen should now look something like thefollowing.
Replacements.Replace private-key-file-path with the path to theprivate-key.pem file that corresponds to your development endpoint's publickey.Replace ec2-12-345-678-9.compute-1.amazonaws.com withthe public address of your development endpoint. You can find the public address intheAWS Glue console by choosing Dev endpoints. Then choose the name of thedevelopment endpoint to open its Endpoint details page.Running Your Script on theDevelopment EndpointTo run your script on your development endpoint, open another terminal window thatsupports SSH, and type this command with the replacements that follow.
Replacements.Replace private-key-file-path with the path to theprivate-key.pem file that corresponds to your development endpoint's publickey.Replace ec2-12-345-678-9.compute-1.amazonaws.com withthe public address of your development endpoint. You can find the public address intheAWS Glue console by navigating to Dev endpoints.
Then choose the name ofthe development endpoint to open its Endpoint details page.Replace deployed-script-path with the path that youentered in the Deployment Mappings tab (for example,/home/glue/scripts/legislators/).Replace script-name with the name of the script thatyou uploaded (for example, getpersonschema.py).PyCharm now prompts you to provide a local source file equivalent to the one beingdebuggedremotely.
Now, click 'Create' and you will have a PyCharm application that will run using the interpreter of the raspberry pi Setting up PostgreSQLNow that you've set up your Pi, you'll want to set up the postgreSQL.To easily do this from the IDE, simply press command + shift + a (ctrl + shift + a on the PC). Then enter Start SSH Session and choose the option that is presented in the list. You will then need to choose the host to connect to. Simply choose the option that you initially created when creating the remote host at the beginning of the tutorial.
Open an SQL console to enter the commands to create a table that will store ping information. Enter the following information: create table pings(recordedat timestamp with time zone default now,destination text,ttl integer,bytesreceived integer,pingtime numeric);create index pingsrecordedat on pings(recordedat);Now, you'll need to modify your ping script to be a little more robust. Navigate to the following github link to find the codeAdding the log fileEnter the following to add a log file that your new py script will deposit log information into. Sudo touch /var/log/raspberryping.logsudo chown pi:pi /var/log/raspberryping.logCrontabApparently RaspberryPi has this cool program scheduler, so we are going to utilize it to run the ping program.
SSH back into the Raspberry Pi and run the command crontab -e to open up cron file. If this is the first time you're opening the file, you will be asked to create a new one.
Also, you'll be asked which editor you prefer to use. Add this information to the end of the file./5. /home/pi/raspberryping/ping.py jetbrains.com /var/log/raspberryping.log 2&1Verifying that your code worksYou can now open up your postgreSQL console and run:SELECT. FROM pingsYou should see some of the results from your pings.