Difference between revisions of "Main Page"

From PaskvilWiki
Jump to: navigation, search
m (→‎ffmpeg)
(→‎PSQL, Postgres, PostgreSQL)
 
(56 intermediate revisions by one user not shown)
Line 8: Line 8:
  
 
Just write me at josef.at.paskvil.dot.com.
 
Just write me at josef.at.paskvil.dot.com.
−
 
−
== SVN ==
 
−
 
−
=== [[Simple Guide to SVN Client]] ===
 
−
Bullets on how to use SVN client.
 
−
 
−
=== [[Creating a SVN Repository]] ===
 
−
How to create a new SVN repository to serve using <tt>svnserve</tt>, via <tt>svn+ssh</tt> or via Apache.
 
−
 
−
=== [[Moving a SVN Repository]] ===
 
−
Moving complete SVN repository from one host to another.
 
−
 
−
=== [[Setup a SVN Server on a Shared Hosting]] ===
 
−
How to setup SVN hosting (via <tt>svn+ssh</tt>) if you're on a shared hosting client - no root access.
 
  
 
== References and Tutorials ==
 
== References and Tutorials ==
Line 33: Line 19:
 
=== [[Small SPARQL, RDQL, etc. Cheat Sheet]] ===
 
=== [[Small SPARQL, RDQL, etc. Cheat Sheet]] ===
  
−
=== [[Small 4store How-To]] ===
+
== Short Notes - Tips and Tricks ==
 +
 
 +
=== [[Small 4store How-To|4store]] ===
  
 
* setup and installation
 
* setup and installation
Line 40: Line 28:
 
* data import and querying from Python
 
* data import and querying from Python
 
* Turtle file printer (in Python)
 
* Turtle file printer (in Python)
−
 
−
=== [[Small Kyoto Cabinet and Kyoto Tycoon Notes]] ===
 
−
 
−
* setup and install Kyoto Cabinet
 
−
* setup and install Kyoto Tycoon
 
−
* KT's RPC API Reference
 
−
 
−
=== [[Small elasticsearch Notes]] ===
 
−
 
−
* Installation
 
−
* What You Get
 
−
* Indexing, Routing, etc.
 
−
* Deleting
 
−
* Getting and Multi-Getting
 
−
* Updating
 
−
* [[ElasticSearch Search|Search]]
 
−
** [[ElasticSearch Query DSL|Query DSL]]
 
−
 
−
=== [[Small MongoDB Notes]] ===
 
−
 
−
Esp. using the PHP PECL driver.
 
−
 
−
== Short Notes - Tips and Tricks ==
 
  
 
=== [[Short Notes on Apache|Apache]] ===
 
=== [[Short Notes on Apache|Apache]] ===
Line 70: Line 35:
 
* Enable SSL/HTTPS in Apache
 
* Enable SSL/HTTPS in Apache
 
* <tt>.htaccess</tt> and <tt>mod_rewrite</tt> Tricks
 
* <tt>.htaccess</tt> and <tt>mod_rewrite</tt> Tricks
 +
 +
=== [[Short Notes on AWS|AWS]] ===
 +
 +
* Debugging (lack of) connection to instance
 +
* Authorization header being removed by ElasticBeanstalk
 +
* Notes on IAM
 +
* AWS Lambda with Python - Notes
 +
* boto3 Snippets
 +
* AWS Chalice
 +
* Building a Python Layer
  
 
=== [[Short Notes on Bash|Bash]] ===
 
=== [[Short Notes on Bash|Bash]] ===
Line 89: Line 64:
 
$ rm !(pattern|pattern)          - delete files that do not match given patterns
 
$ rm !(pattern|pattern)          - delete files that do not match given patterns
 
$ find . -mtime +5 -exec rm {} \; - delete all files in this folder that are older than 5 days
 
$ find . -mtime +5 -exec rm {} \; - delete all files in this folder that are older than 5 days
−
<Ctrl-u> ... <Ctrl-y>            - Ctrl+u will kill (cut) current command, then you can do whatever you need, and Ctrl+y will paste (yank) the line back
+
<Ctrl-u> ... <Ctrl-y>            - Ctrl+u will cut current command, then you can do whatever you need, and Ctrl+y will paste it back
−
$ date -d @1234567890               - convert unix timestamp to human-readable</pre>
+
$ date -d @1234567890             - convert unix timestamp to human-readable
 +
$ jul () { date -d "$1-01-01 +$2 days -1 day" "+%Y%m%d"; }  - `jul $year $daynum` prints $daynum's day of $year, with Jan 1st being day 1</pre>
 
* ''Informations''
 
* ''Informations''
 
<pre>$ mount | column -t              - pretty-print mount table
 
<pre>$ mount | column -t              - pretty-print mount table
 
$ cat /etc/issue                  - display distro name</pre>
 
$ cat /etc/issue                  - display distro name</pre>
 
* ''Small Tricks''
 
* ''Small Tricks''
−
<pre>$ mount -t tmpfs tmpfs /mnt -o size=1024m      - mount temporary RAM partition (fast access, non-persistent!)
+
<pre>$ echo $(printf '%q' "A\"B")      - prints escaped string (in this case, > A\"B <)
 +
$ mount -t tmpfs tmpfs /mnt -o size=1024m      - mount temporary RAM partition (fast access, non-persistent!)
 
$ disown -a && exit              - exit the shell while keeping all running processes alive
 
$ disown -a && exit              - exit the shell while keeping all running processes alive
 
$ ssh -t remote_host screen -r    - directly connect to a remote screen process
 
$ ssh -t remote_host screen -r    - directly connect to a remote screen process
Line 102: Line 79:
 
$ fuser -k filename              - kill process that is locking given file
 
$ fuser -k filename              - kill process that is locking given file
 
$ readom dev=/dev/scd0 f=/path/to/image.iso    - create CD/DVD ISO image
 
$ readom dev=/dev/scd0 f=/path/to/image.iso    - create CD/DVD ISO image
 +
$ diff -rq folder-1 folder-2      - compare 2 folders (files missing or differing)
 
$ mkdir -p a/deep/dir/tree        - create nested directories fast
 
$ mkdir -p a/deep/dir/tree        - create nested directories fast
 
$ mount file.iso /mnt/iso -o loop - mount ISO file as drive /mnt/iso (dir has to preexist)
 
$ mount file.iso /mnt/iso -o loop - mount ISO file as drive /mnt/iso (dir has to preexist)
−
$ sudo touch /forcefsck          - force a file system check on the next reboot</pre>
+
$ sudo touch /forcefsck          - force a file system check on the next reboot
 +
$ ls | while read f; do process_file "$f"; done    - iterate through files even if they contain space in name
 +
$ while read f; do process_file "$f"; done < <(ls) - iterates through files, but does not create sub-process (outside scope is visible)</pre>
 
* ''lsof''
 
* ''lsof''
 
<pre>$ lsof -nPi                      - list network connections
 
<pre>$ lsof -nPi                      - list network connections
Line 111: Line 91:
 
$ lsof +D [folder]                - discover all open files in the [folder]
 
$ lsof +D [folder]                - discover all open files in the [folder]
 
$ lsof -p [pid]                  - list files opened by given PID
 
$ lsof -p [pid]                  - list files opened by given PID
−
$ lsof -i :8080                  - list programs that have port 8080 open ("who to kill" if you get "port already in use")</pre>
+
$ lsof -i :8080                  - list programs that have port 8080 open ("who to kill" if you get "port already in use")
 +
$ if [ `lsof -- file | wc -l` -ne 0 ] ... - tests if file is open by some process</pre>
  
 
=== [[Short Notes on C/C++|C/C++]] ===
 
=== [[Short Notes on C/C++|C/C++]] ===
Line 119: Line 100:
 
* Shared Libraries on Linux+GCC
 
* Shared Libraries on Linux+GCC
 
* Singleton
 
* Singleton
 +
* <tt>std::move</tt> Semantics
 +
* Converting between <tt>std::string</tt> and <tt>std::wstring</tt> in C++11
 +
 +
=== cURL (in terminal) ===
 +
 +
Use '''-s''' option to make <tt>curl</tt> silent.
 +
 +
* POST to server using cURL:
 +
<pre>curl --data "key1=value1&key2=value2" http://www.example.com/post.php</pre>
 +
* Upload to FTP server (with credentials), having connection timeout 60 seconds, and 120 seconds maximum operation time:
 +
<pre>curl --connect-timeout 60 -m 120 -T file.ext "ftp://12.34.56.78/folder" --user user:pass</pre>
 +
* Download from FTP server (with credentials):
 +
<pre>curl --user user:pass "ftp://12.34.56.78/folder/file.ext" -o ~/Downloads/file.ext</pre>
 +
 +
=== [[Small Docker Notes|Docker]] ===
 +
 +
=== [[Small elasticsearch Notes|ElasticSearch]] ===
 +
 +
* Installation
 +
* What You Get
 +
* Indexing, Routing, etc.
 +
* Deleting
 +
* Getting and Multi-Getting
 +
* Updating
 +
* Search and Query DSL
  
 
=== [[Short Notes on ffmpeg|ffmpeg]] ===
 
=== [[Short Notes on ffmpeg|ffmpeg]] ===
 +
 +
* ''process all files, may include spaces in file name:''
 +
ls *.avi | while read f; do ffmpeg -i "$f" ''...params...'' "$f.mkv"; done
 +
* ''if you're having problems with input format, passing video through mpeg typically helps:''
 +
ffmpeg -i file.mkv -qscale 2 -f mpeg - | ffmpeg -i - ''...params...'' file.avi
 +
* ''downmix 5.1 dolby to normal stereo:''
 +
ffmpeg -i file.mkv ''...params...'' -acodec libmp3lame -ar 44100 -ac 2 file.avi
 +
* ''add an overlay image to the video:''
 +
ffmpeg -i input.mp4 -i image.png -filter_complex "[0:v][1:v] overlay=''x'':''y'':enable='between(t,0,20)'" -pix_fmt yuv420p -c:a copy output.mp4
 +
This will add ''image.png'' as an overlay at position ''x'', ''y'' over the ''imput.mp4'' video, for the first 20 seconds.
 +
* ''fade in and out, first and last 25 frames of a 1000 frame video''
 +
ffmpeg -i input.mp4 -vf "fade=in:0:25,fade=out:975:25" -acodec copy out.mp4
 +
* ''convert album contained in APE or similar file with CUE into separate MP3 tracks''
 +
ffmpeg -i cd.ape -ab 256k cd.mp3
 +
mp3splt -a -c cd.cue cd.mp3
 +
 +
==== [[Short Notes on ffmpeg|more on ffmpeg]] ====
  
 
* Rip DVD into FLV (or anything else)
 
* Rip DVD into FLV (or anything else)
−
* Still Image with Audio
+
* Video File with Still Image and Audio
 +
* Concatenate Videos
 +
 
 +
=== [[Short Notes on Flask and Flask-RestPlus|Flask and Flask-RestPlus]] ===
 +
 
 +
=== [[Short Notes on git|git]] ===
 +
 
 +
=== [[Short Notes on JS|All things JS - React, Vue, ...]] ===
 +
 
 +
=== [[Small Kyoto Cabinet and Kyoto Tycoon Notes|Kyoto Cabinet and Kyoto Tycoon]] ===
 +
 
 +
* Setup and Install Kyoto Cabinet
 +
* Setup and Install Kyoto Tycoon
 +
* KT's RPC API Reference
  
 
=== [[Short Notes on Linux Administration|Linux Administration]] ===
 
=== [[Short Notes on Linux Administration|Linux Administration]] ===
Line 131: Line 167:
 
* Installing <tt>sendmail</tt>
 
* Installing <tt>sendmail</tt>
 
* Sending Emails using <tt>sendmail</tt>
 
* Sending Emails using <tt>sendmail</tt>
 +
* Find and Mark Bad Blocks
 +
* Using TLP to optimize battery lifetime
  
 
=== [[Short Notes on MAC OSX|MAC OSX]] ===
 
=== [[Short Notes on MAC OSX|MAC OSX]] ===
Line 144: Line 182:
 
  $ (just once) sudo port install port_cutleaves
 
  $ (just once) sudo port install port_cutleaves
 
  $ sudo port_cutleaves
 
  $ sudo port_cutleaves
 +
 +
=== [[Small MongoDB Notes|MongoDB]] ===
 +
 +
* Installation on Ubuntu
 +
* Troubleshooting on Desktop
 +
* Use in PHP via PECL driver
 +
 +
=== [[Short Notes on MySQL|MySQL]] ===
 +
 +
* Error 1045: Access denied
 +
* The <tt>insert ... on duplicate key update</tt> Pitfall
 +
* Shrink ''ibfile1'' InnoDB File
 +
* Index JSON columns
  
 
=== [[Short Notes on PDF|PDF]] ===
 
=== [[Short Notes on PDF|PDF]] ===
Line 158: Line 209:
 
* Prevent Client Disconnect from Killing PHP
 
* Prevent Client Disconnect from Killing PHP
 
* Disconnect Client and Continue Processing
 
* Disconnect Client and Continue Processing
 +
 +
=== [[Short Notes on PSQL|PSQL, Postgres, PostgreSQL]] ===
 +
 +
* Dump and Restore
 +
* Delete Duplicate Rows
 +
* Set Sequence Value to Max of Table's ID
  
 
=== [[Short Notes on Python|Python]] ===
 
=== [[Short Notes on Python|Python]] ===
  
 
* Timing, and memory, on Linux
 
* Timing, and memory, on Linux
 +
* Importing Files
 +
* uWSGI, nginx, Flask
 +
* Decorators
 +
* virtualenv
 +
* SQLAlchemy
 +
* iPython fails with ''NameError: name 'sys' is not defined'' on autocompletion
 +
 +
=== [[RedBeanPHP Cheat Sheet]] ===
 +
 +
=== [[Short Notes on RESTful APIs|RESTful APIs]] ===
  
 
=== [[Short Notes on Security|Security]] ===
 
=== [[Short Notes on Security|Security]] ===
Line 168: Line 235:
 
* SSH Access using pubkey's Authentication
 
* SSH Access using pubkey's Authentication
 
* Using RSync together with SSH
 
* Using RSync together with SSH
 +
 +
=== [[Short Notes on SVN|Subversion/SVN]] ===
 +
 +
* Simple Guide to SVN Client
 +
* Creating a SVN Repository
 +
* Moving a SVN Repository
 +
* Change Repository URL (Subversion 1.7+)
 +
* Setup a SVN Server on a Shared Hosting with no ''root'' Access
 +
* creating and applying patch (the <tt>--diff-cmd</tt> is necessary in case you set some graphical diff-viewer in <tt>~/.subversion/config</tt>):
 +
<pre>cd root_of_project
 +
svn di --diff-cmd diff > ~/patch.diff
 +
cd root_of_other_project
 +
patch -p0 -i ~/patch.diff</pre>
 +
* ''svn:externals'' - pull ''repo1'' into ''dir1'', and ''repo2'' into ''dir2'':
 +
<pre>echo -e 'dir1 svn://repo1\ndir2 svn://repo2' > repos.txt
 +
svn propset svn:externals . -F repos.txt</pre>
 +
validate with " ''svn propget svn:externals .'' " .
 +
 +
=== Ubuntu / Debian / Gnome etc. ===
 +
 +
* restrict gnome app switcher to current workspace only:
 +
<pre>gsettings set org.gnome.shell.app-switcher current-workspace-only true</pre>
 +
* restore or undelete (config) file or directory:
 +
<pre># 1. check what package file or folder belongs to
 +
$ dpkg -S /path/my.conf
 +
# 2.a replace the config file; if it exists, will offer diff
 +
$ sudo apt-get -o Dpkg::Options::="--force-confask" install --reinstall <package-name>
 +
# 2.b restore the directory
 +
$ sudo apt-get -o Dpkg::Options::="--force-confask" install --reinstall $(dpkg -S /etc/some/directory | sed 's/,//g; s/:.*//')</pre>
 +
* increase number of inotify watchers
 +
<pre>$ cat /proc/sys/fs/inotify/max_user_watches
 +
8192
 +
$ echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
 +
$ sudo sysctl -p
 +
$ cat /proc/sys/fs/inotify/max_user_watches
 +
524288</pre>
 +
 +
==== Up-to-Date Versions of Packages ====
 +
 +
* '''Python'''
 +
Ubuntu 18.04 ships with Python 3.6, but Amazon Linux uses 3.7, so e.g. to use and build Lambda layers and match those against packages you deploy, you need to install 3.7:
 +
<pre>$ sudo add-apt-repository ppa:deadsnakes/ppa
 +
$ sudo apt update
 +
$ sudo apt install python3.7 python3.7-venv</pre>
 +
Since these do not override default <tt>python</tt> and <tt>python3</tt> links (and better not!), you should create corresponding venv's as:
 +
<pre>$ python3.7 -m venv [env-dir]</pre>
 +
 +
* '''NPM, node.js, etc.'''
 +
The default <tt>apt</tt> ''npm'' version is awfully outdated.
 +
 +
'''update''' - available as a ''snap'' as well, no need to add ''apt'' sources - https://node.dev/node-binary
 +
 +
Add NodeSource APT repo; for other versions, e.g. 12.x, change to <tt>setup_12.x</tt>.
 +
<pre>$ curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
 +
$ sudo apt update
 +
$ sudo apt install nodejs</pre>
  
 
=== [[Short Notes on ViM|ViM]] ===
 
=== [[Short Notes on ViM|ViM]] ===
Line 179: Line 302:
 
$ vim +10 [file] ; vim +/hello [file]  - open [file] at line 10, or at first occurrence of pattern 'hello' in file
 
$ vim +10 [file] ; vim +/hello [file]  - open [file] at line 10, or at first occurrence of pattern 'hello' in file
 
:w !sudo tee %                        - save file via `sudo` (as root)</pre>
 
:w !sudo tee %                        - save file via `sudo` (as root)</pre>
 +
 +
=== [[Short Notes on Wavelets|Wavelets]] ===
 +
 +
* Integer Haar Wavelets in Python
  
 
=== [[Short Notes on Web|Web - HTML, CSS, JS, jQuery, etc.]] ===
 
=== [[Short Notes on Web|Web - HTML, CSS, JS, jQuery, etc.]] ===
Line 193: Line 320:
  
 
==== [[Install Apache and PHP to do Secure h264 Pseudo Streaming]] ====
 
==== [[Install Apache and PHP to do Secure h264 Pseudo Streaming]] ====
 +
 +
==== HTTP Response Codes ====
 +
 +
This is not a definitive list, it's just stuff I use the most, and always forget which is which.
 +
 +
* '''100''' - ''Continue''
 +
* '''200''' - ''OK''
 +
* '''201''' - ''Created''
 +
* '''202''' - ''Accepted''
 +
* '''204''' - ''No Content''
 +
* '''301''' - ''Moved Permanently''
 +
* '''302''' - ''Found'' (I typically use this instead of the '''307''' "''Temporary Redirect''")
 +
* '''304''' - ''Not Modified''
 +
* '''400''' - ''Bad Request''
 +
* '''401''' - ''Unauthorized''
 +
* '''403''' - ''Forbidden''
 +
* '''404''' - ''Not Found''
 +
* '''405''' - ''Method Not Allowed'' (using GET on POST calls, etc. - must include "Allow" response header)
 +
* '''406''' - ''Not Acceptable'' (mismatch of response format and "Accept" headers)
 +
* '''500''' - ''Internal Server Error''
 +
* '''501''' - ''Not Implemented''
 +
* '''503''' - ''Service Unavailable''

Latest revision as of 16:34, 9 July 2021

Welcome to the PaskvilWiki!

The future shape of this wiki is unknown for now, let's hope it'll be something good...

This is partially a private wiki - viewing is allowed to all, but editing is allowed only to registered users.

This is more of a prevention than denial - I don't have time to monitor and moderate edits; - you're more than welcome to request registration, all contributors are very much welcome!

Just write me at josef.at.paskvil.dot.com.

References and Tutorials

Emacs and Slime Reference

Small jQuery Reference

Small jQuery Cheat Sheet

Small SPARQL, RDQL, etc. Cheat Sheet

Short Notes - Tips and Tricks

4store

  • setup and installation
  • running 4store
  • data import
  • data import and querying from Python
  • Turtle file printer (in Python)

Apache

  • Note on serving of local files
  • Self-Signed SSL Certificate
  • Enable SSL/HTTPS in Apache
  • .htaccess and mod_rewrite Tricks

AWS

  • Debugging (lack of) connection to instance
  • Authorization header being removed by ElasticBeanstalk
  • Notes on IAM
  • AWS Lambda with Python - Notes
  • boto3 Snippets
  • AWS Chalice
  • Building a Python Layer

Bash

  • Extensions and File Name
  • File Test Operators
  • Commands and Shortcuts 
$ sudo !!                         - run last command as root
$ !!:gs/foo/bar                   - run last command, replacing 'foo' with 'bar'
$ ^foo^bar                        - run last command, replacing 'foo' with 'bar'
$ cp filename{,.bak}              - create a .bak copy of file
<Ctrl-x> <Ctrl-e>                 - open $EDITOR to edit current command line; upon exit, content will be run
$ <space>command                  - run the command without saving it in history
$ > file.txt                      - empty a file
<Alt+.>                           - put last argument on cursor place; repeat to cycle through
$ (cd /tmp && ls)                 - jump to /tmp, execute ls, and jump back here
$ pushd /tmp .... $ popd          - goes to /tmp, remembering current location; at later time, popd will bring you back
$ \command                        - run command without using aliases
$ rm !(pattern|pattern)           - delete files that do not match given patterns
$ find . -mtime +5 -exec rm {} \; - delete all files in this folder that are older than 5 days
<Ctrl-u> ... <Ctrl-y>             - Ctrl+u will cut current command, then you can do whatever you need, and Ctrl+y will paste it back
$ date -d @1234567890             - convert unix timestamp to human-readable
$ jul () { date -d "$1-01-01 +$2 days -1 day" "+%Y%m%d"; }   - `jul $year $daynum` prints $daynum's day of $year, with Jan 1st being day 1
  • Informations 
$ mount | column -t               - pretty-print mount table
$ cat /etc/issue                  - display distro name
  • Small Tricks 
$ echo $(printf '%q' "A\"B")      - prints escaped string (in this case, > A\"B <)
$ mount -t tmpfs tmpfs /mnt -o size=1024m      - mount temporary RAM partition (fast access, non-persistent!)
$ disown -a && exit               - exit the shell while keeping all running processes alive
$ ssh -t remote_host screen -r    - directly connect to a remote screen process
$ for I in $(mysql -e 'show databases' -s --skip-column-names); do mysqldump $I | gzip > "$I.sql.gz"; done
                                  - backup all databases into individual files
$ fuser -k filename               - kill process that is locking given file
$ readom dev=/dev/scd0 f=/path/to/image.iso    - create CD/DVD ISO image
$ diff -rq folder-1 folder-2      - compare 2 folders (files missing or differing)
$ mkdir -p a/deep/dir/tree        - create nested directories fast
$ mount file.iso /mnt/iso -o loop - mount ISO file as drive /mnt/iso (dir has to preexist)
$ sudo touch /forcefsck           - force a file system check on the next reboot
$ ls | while read f; do process_file "$f"; done    - iterate through files even if they contain space in name
$ while read f; do process_file "$f"; done < <(ls) - iterates through files, but does not create sub-process (outside scope is visible)
  • lsof 
$ lsof -nPi                       - list network connections
$ netstat -tlnp                   - print all listening ports with PID's (run as root to see all process names)
$ lsof -c gnome-terminal          - list files opened by given command
$ lsof +D [folder]                - discover all open files in the [folder]
$ lsof -p [pid]                   - list files opened by given PID
$ lsof -i :8080                   - list programs that have port 8080 open ("who to kill" if you get "port already in use")
$ if [ `lsof -- file | wc -l` -ne 0 ] ... - tests if file is open by some process

C/C++

  • How to clean up after child thread
  • Non-blocking IO using sockets
  • Shared Libraries on Linux+GCC
  • Singleton
  • std::move Semantics
  • Converting between std::string and std::wstring in C++11

cURL (in terminal)

Use -s option to make curl silent.

  • POST to server using cURL: 
curl --data "key1=value1&key2=value2" http://www.example.com/post.php
  • Upload to FTP server (with credentials), having connection timeout 60 seconds, and 120 seconds maximum operation time: 
curl --connect-timeout 60 -m 120 -T file.ext "ftp://12.34.56.78/folder" --user user:pass
  • Download from FTP server (with credentials): 
curl --user user:pass "ftp://12.34.56.78/folder/file.ext" -o ~/Downloads/file.ext

Docker

ElasticSearch

  • Installation
  • What You Get
  • Indexing, Routing, etc.
  • Deleting
  • Getting and Multi-Getting
  • Updating
  • Search and Query DSL

ffmpeg

  • process all files, may include spaces in file name: 
ls *.avi | while read f; do ffmpeg -i "$f" ...params... "$f.mkv"; done
  • if you're having problems with input format, passing video through mpeg typically helps: 
ffmpeg -i file.mkv -qscale 2 -f mpeg - | ffmpeg -i - ...params... file.avi
  • downmix 5.1 dolby to normal stereo: 
ffmpeg -i file.mkv ...params... -acodec libmp3lame -ar 44100 -ac 2 file.avi
  • add an overlay image to the video: 
ffmpeg -i input.mp4 -i image.png -filter_complex "[0:v][1:v] overlay=x:y:enable='between(t,0,20)'" -pix_fmt yuv420p -c:a copy output.mp4

This will add image.png as an overlay at position x, y over the imput.mp4 video, for the first 20 seconds.

  • fade in and out, first and last 25 frames of a 1000 frame video 
ffmpeg -i input.mp4 -vf "fade=in:0:25,fade=out:975:25" -acodec copy out.mp4
  • convert album contained in APE or similar file with CUE into separate MP3 tracks 
ffmpeg -i cd.ape -ab 256k cd.mp3
mp3splt -a -c cd.cue cd.mp3

more on ffmpeg

  • Rip DVD into FLV (or anything else)
  • Video File with Still Image and Audio
  • Concatenate Videos

Flask and Flask-RestPlus

git

All things JS - React, Vue, ...

Kyoto Cabinet and Kyoto Tycoon

  • Setup and Install Kyoto Cabinet
  • Setup and Install Kyoto Tycoon
  • KT's RPC API Reference

Linux Administration

  • Reset Compiz to Default Settings
  • Enable and Start sendmail
  • Installing sendmail
  • Sending Emails using sendmail
  • Find and Mark Bad Blocks
  • Using TLP to optimize battery lifetime

MAC OSX

  • Asynchronous Execution
    • NSThread
    • performSelectorInBackground
    • dispatch_async
  • MacPorts 
$ sudo port selfupdate
$ sudo port upgrade outdated
$ sudo port uninstall inactive
$ (just once) sudo port install port_cutleaves
$ sudo port_cutleaves

MongoDB

  • Installation on Ubuntu
  • Troubleshooting on Desktop
  • Use in PHP via PECL driver

MySQL

  • Error 1045: Access denied
  • The insert ... on duplicate key update Pitfall
  • Shrink ibfile1 InnoDB File
  • Index JSON columns

PDF

  • Merging PDF Files using GhostScript
  • Cutting PDF Files using GhostScript
  • Converting to PDF using GhostScript

PHP

  • Convert Accented Characters to Non-Accented
  • Upload a File using cURL
  • HTTP Post in Pure PHP (without cURL)
  • Prevent Client Disconnect from Killing PHP
  • Disconnect Client and Continue Processing

PSQL, Postgres, PostgreSQL

  • Dump and Restore
  • Delete Duplicate Rows
  • Set Sequence Value to Max of Table's ID

Python

  • Timing, and memory, on Linux
  • Importing Files
  • uWSGI, nginx, Flask
  • Decorators
  • virtualenv
  • SQLAlchemy
  • iPython fails with NameError: name 'sys' is not defined on autocompletion

RedBeanPHP Cheat Sheet

RESTful APIs

Security

  • Password Generators
  • SSH Access using pubkey's Authentication
  • Using RSync together with SSH

Subversion/SVN

  • Simple Guide to SVN Client
  • Creating a SVN Repository
  • Moving a SVN Repository
  • Change Repository URL (Subversion 1.7+)
  • Setup a SVN Server on a Shared Hosting with no root Access
  • creating and applying patch (the --diff-cmd is necessary in case you set some graphical diff-viewer in ~/.subversion/config): 
cd root_of_project
svn di --diff-cmd diff > ~/patch.diff
cd root_of_other_project
patch -p0 -i ~/patch.diff
  • svn:externals - pull repo1 into dir1, and repo2 into dir2: 
echo -e 'dir1 svn://repo1\ndir2 svn://repo2' > repos.txt
svn propset svn:externals . -F repos.txt

validate with " svn propget svn:externals . " .

Ubuntu / Debian / Gnome etc.

  • restrict gnome app switcher to current workspace only: 
gsettings set org.gnome.shell.app-switcher current-workspace-only true
  • restore or undelete (config) file or directory: 
# 1. check what package file or folder belongs to
$ dpkg -S /path/my.conf
# 2.a replace the config file; if it exists, will offer diff
$ sudo apt-get -o Dpkg::Options::="--force-confask" install --reinstall <package-name>
# 2.b restore the directory
$ sudo apt-get -o Dpkg::Options::="--force-confask" install --reinstall $(dpkg -S /etc/some/directory | sed 's/,//g; s/:.*//')
  • increase number of inotify watchers 
$ cat /proc/sys/fs/inotify/max_user_watches 
8192
$ echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
$ sudo sysctl -p
$ cat /proc/sys/fs/inotify/max_user_watches 
524288

Up-to-Date Versions of Packages

  • Python

Ubuntu 18.04 ships with Python 3.6, but Amazon Linux uses 3.7, so e.g. to use and build Lambda layers and match those against packages you deploy, you need to install 3.7: 

$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt update
$ sudo apt install python3.7 python3.7-venv

Since these do not override default python and python3 links (and better not!), you should create corresponding venv's as: 

$ python3.7 -m venv [env-dir]
  • NPM, node.js, etc.

The default apt npm version is awfully outdated.

update - available as a snap as well, no need to add apt sources - https://node.dev/node-binary

Add NodeSource APT repo; for other versions, e.g. 12.x, change to setup_12.x. 

$ curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
$ sudo apt update
$ sudo apt install nodejs

ViM

  • Delete all lines matching a pattern
  • Directly type HTML entities
  • Reformat a plain text paragraph 
$ vim -x filename                      - open filename for editing, and save it encrypted (will prompt for a key)
:X                                     - set encryption for current file; use ':set key=' to disable encryption
                                       - put "set cm=blowfish" in your .vimrc to enable safer encryption by default
$ vim +10 [file] ; vim +/hello [file]  - open [file] at line 10, or at first occurrence of pattern 'hello' in file
:w !sudo tee %                         - save file via `sudo` (as root)

Wavelets

  • Integer Haar Wavelets in Python

Web - HTML, CSS, JS, jQuery, etc.

  • Multiple Submit Buttons and/or Multiple Forms on Page
  • Make 'float' Elements Appear One under Another
  • Upload a File via Form
  • Override the IE's Problem with width
  • Make an Element Fill In All the Remaining Space

Other Short Notes

Rip Video DVD that is not Finalized

Install Apache and PHP to do Secure h264 Pseudo Streaming

HTTP Response Codes

This is not a definitive list, it's just stuff I use the most, and always forget which is which.

  • 100 - Continue
  • 200 - OK
  • 201 - Created
  • 202 - Accepted
  • 204 - No Content
  • 301 - Moved Permanently
  • 302 - Found (I typically use this instead of the 307 "Temporary Redirect")
  • 304 - Not Modified
  • 400 - Bad Request
  • 401 - Unauthorized
  • 403 - Forbidden
  • 404 - Not Found
  • 405 - Method Not Allowed (using GET on POST calls, etc. - must include "Allow" response header)
  • 406 - Not Acceptable (mismatch of response format and "Accept" headers)
  • 500 - Internal Server Error
  • 501 - Not Implemented
  • 503 - Service Unavailable
Retrieved from "https://wiki.paskvil.com/index.php?title=Main_Page&oldid=368"