Daily Snippet, Developer to developer

Publish CSV to the world

Enter https://data.world to get your CSV online once you upload it.

Sebelumnya saya ingin menceritakan apa kebutuhan saya sampai harus mempublish CSV. Saya ingin mencari data travel atau biro perjalanan umroh resmi dari kemenag di sekitar Surabaya. Dari googling dapat alamat  https://haji.kemenag.go.id/v3/basisdata/daftar-ppiu.

Datanya berbentuk tabel sederhana. Ada kolom Pencarian tapi nampaknya jika diketik “surabaya” yang tampil hanya Penyelenggara yang mengandung kata “surabaya”. Padahal alamatnya bukan Surabaya. Hmm..terus gimana cara termudahnya, no code, using existing tools only?.

Bisa kita manual Find in page mencari kata “surabaya”, tapi setelah mendapat alamat kita harus manual select text dan copy-paste di excel atau word editor lainnya.

querycaridaftarumrohkemenag.png

Scrape it. Menggunakan tools Chrome extension Data Miner : https://chrome.google.com/webstore/detail/data-scraper-easy-web-scr/nndknepjnldbdbepjfgmncbggmopgden. Dengan Data Miner kita dapat mendownload data CSV sebuah table dengan mudah. Berbekal sedikit tutorial dari youtube saya berhasil mendapat CSV-nya.

Create New Dataset di data.world dan upload CSV. Untuk versi free kita dapat mengupload 100 MB/dataset. Set as public dan anda-pun bisa membukanya disini : https://data.world/anangbakti/travelumrohdepag.

Lebih detil : https://data.world/anangbakti/travelumrohdepag/workspace/file?filename=daftarumroh.csv.

Di tampilan ini kita bisa melakukan query dengan klik tombol New Query. Sebagai contoh saya melakukan pencarian dengan alamat Surabaya, Sidoarjo, dan Mojokerto.

querydatasetumroh.png

Hasil pencarian bisa di-download lagi sebagai CSV.

Tentu saja ada kelemahannya yaitu data tidak update, karena jika data di Kemenag berubah maka perlu scrape manual kembali menggunakan Data Miner untuk diupload lagi ke https://data.world.

Happy query …

Advertisements
Standard

Snap 2017-09-08 at 22.06.39

Daily Snippet, Developer to developer

First run Flutter

Image
Daily Snippet, Developer to developer

Installing moodle 2

I use this tutorial : https://docs.moodle.org/26/en/Step-by-step_Installation_Guide_for_Ubuntu and jump to step 4.

On step 7, after chmod -R 777 /var/www/moodle, you have to open http://<your-ip>/moodle on browser for next installation.

i’ve got this error

server-checks-php5-curl

showing that you don’t have php5-curl, php5-xmlrpc, and php5-intl. Do apt-get install php5-curl php5-xmlrpc php5-intl.

I skip for opcache-enable and click next. moodle-server-min-requirements

and moodle will automatically install all modules moodle-install-all-modules-success

Run to next step on  https://docs.moodle.org/26/en/Step-by-step_Installation_Guide_for_Ubuntu, Database setting and continue to Suggestions: Enable Zend OpCache/Change Document Root.

moodle-after-instal

Moodle is somehow slow, you can use php cache and mysql tuning suggestion on http://moodurian.blogspot.com/2011/07/how-i-speeded-up-my-moodle-site.html.

Basicly you can use : https://raw.github.com/major/MySQLTuner-perl/master/mysqltuner.pl for mysql tuning. It will give you some good recomendation to my.cnf.

and : apt-get install php-apc

to cache everything.

Standard
Daily Snippet, Developer to developer

Basic installation nginx+php5-fpm+yii

As a reference I use tutorial from rosehosting.

  • apt-get update
  • apt-get install nginx
  • Open your browser, http://localhost and you can see nginx welcome page.
  • apt-get install php5-fpm php5-cli php5-mcrypt php5-gd // I ignore install mysql, planned doing later
  • Modify /etc/php5/fpm/php.ini just like on tutorial.
  • Modify nginx conf. nano /etc/nginx/sites-available/default, replace inside server{}

server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;

root /usr/share/nginx/html;
index index.html index.htm;

# Make site accessible from http://localhost/
server_name localhost;
set $yii_bootstrap “index.php”;
charset utf-8;

location / {
index index.html $yii_bootstrap
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ /$yii_bootstrap?args;
# Uncomment to enable naxsi on this location
# include /etc/nginx/naxsi.rules
}

location ~ ^/(protected|framework|themes/\w+/views) {
deny  all;
}

location ~ \.(js|css|png|jpg|gif|swf|ico|pdf|mov|fla|zip|rar)$ {
try_files $uri =404;
}

location ~ \.php {
fastcgi_index index.php;
fastcgi_split_path_info ^(.+\.php)(.*)$;
set $fsn /$yii_bootstrap;
if (-f $document_root$fastcgi_script_name){
set $fsn $fastcgi_script_name;
}
include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}

location ~ /\. {
deny all;
access_log off;
log_not_found off;
}

}

Standard
Developer to developer

Playing yowsup2 and whatsapi

This yowsup2 installation requirement is valid. But I found I had to apt-get install python2.7-dev too, before I can pip install Pillow. Next you can register your number using yowsup-cli. Now test send messages using this command yowsup-cli demos -c -y. Type /help for all command and /L to quick login. Type /message send <number-to> <your-message-here>.

WhatsAPI is more easy to install but I wanna share simple .php test. First git clone the source from github to your /var/www/, and after downloaded, place this php file on /var/www/WhatsApi-Official/.

<?php
require 'src/whatsprot.class.php';
require 'src/events/MyEvents.php';
$username = "your-phone-number ex.62811987654";
$identity = "your-phone-number";
$nickname = "your-phone-number";
$password = "your-password"; // The one we got registering the number
$debug = true;

// Create a instance of WhastPort.
$w = new WhatsProt($username, $identity, $nickname, $debug);
connect(); // Connect to WhatsApp network
$w->loginWithPassword($password); // logging in with the password we got!

//Send Messages
$target = ‘to-phone-number’; // The number of the person you are sending the message ex.6285798766543
$message = ‘Hi! 🙂 this is a test message’;
$w->sendMessage($target , $message);

//Receive Messages
$events = new MyEvents($w);
$events->setEventsToListenFor($events->activeEvents);
$w->pollMessage();
?>

Standard