mac tensorflow jupyter notebook 환경설정 기록

virtualenv, virtualwrapper 설치 사이트

https://beomi.github.io/2016/12/28/HowToSetup-Virtualenv-VirtualenvWrapper/

 

최초 터미널 접속후 . .bashrc를 실행

  1. 가상환경 목록 확인
    1. workon
  2. 가상환경 접속
    1. workon vision_env
  3. 가상환경 생성
    1. mkvirtualenv env_name
  4. 가상환경 제거
    1. rmvirtualenv env_name

 

 

문제1

ModuleNotFoundError: No module named 'sklearn'

해결책 1

pip3 install -U scikit-learn scipy matplotlib

 

 

Advertisements

mac tensorflow error

맥 (10.13.6) 에서 주피터 노트북으로 텐서플로 1.10 설치한 후 텐서플로만 import 하면 아래처럼 에러가 빽 뜸.

The kernel appears to have died. It will restart automatically. tensorflow on jupyter

 

한참 헤매다가 파이썬에서 직접 텐서플로 import 만 해도 아래처럼 에러가 나고있다

> import tensorflow

Illegal instruction: 4

 

한참 찾아보니 비슷하게 고통받는 사람들이 많았다
해결책은 tensorflow 1.5 버전으로 내리는 것뿐이라고 한다.

(방법)
pip uninstall tensorflow
pip install tensorflow==1.5

(테스트 기록)
1.5 : 성공

Successfully installed tensorflow-1.5.0

(vision_env) [~/.virtualenvs/vision_env/bin]$ python

Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54)

[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin

Type “help”, “copyright”, “credits” or “license” for more information.

>>> import tensorflow as tf

>>> print(tf.__version__)

1.5.0

>>>

1.6 : 실패

Successfully installed tensorboard-1.6.0 tensorflow-1.6.0
(vision_env) [~/.virtualenvs/vision_env/bin]$ python
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import tensorflow as tf
Illegal instruction: 4
(vision_env) [~/.virtualenvs/vision_env/bin]$

1.7 : 실패

Successfully installed tensorboard-1.7.0 tensorflow-1.7.0

(vision_env) [~/.virtualenvs/vision_env/bin]$ python

Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54)

[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin

Type “help”, “copyright”, “credits” or “license” for more information.

>>> import tensorflow as tf

Illegal instruction: 4

(vision_env) [~/.virtualenvs/vision_env/bin]$

1.8 : 실패

Successfully installed tensorboard-1.8.0 tensorflow-1.8.0

(vision_env) [~/.virtualenvs/vision_env/bin]$ python

Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54)

[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin

Type “help”, “copyright”, “credits” or “license” for more information.

>>> import tensorflow as tf

Illegal instruction: 4

(vision_env) [~/.virtualenvs/vision_env/bin]$

1.9 : 실패

Successfully installed tensorboard-1.9.0 tensorflow-1.9.0

(vision_env) [~/.virtualenvs/vision_env/bin]$ python

Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54)

[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin

Type “help”, “copyright”, “credits” or “license” for more information.

>>> import tensorflow as tf

Illegal instruction: 4

(vision_env) [~/.virtualenvs/vision_env/bin]$

1.10 : 실패

Successfully installed tensorboard-1.10.0 tensorflow-1.10.0

(vision_env) [~/.virtualenvs/vision_env/bin]$ python

Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54)

[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin

Type “help”, “copyright”, “credits” or “license” for more information.

>>> import tensorflow as tf

Illegal instruction: 4

(vision_env) [~/.virtualenvs/vision_env/bin]$

 

(주피터 노트북에서의 테스트)
tensorflow : 1.5 사용

20180815_001

성공…

늘 환경구성에서 힘을 다 뺀다…

flyway 명령어 정리

  1. 다른 계정으로 flyway에 접속하여 수행
    • flyway -url=jdbc:mariadb://127.0.0.1:3306/aivapp -user=root -password=passwd -target=01.00 migrate
  2. flyway에서 관리된 sql 스크립트 목록 확인
    • flyway info
  3. sql 스크립트 수행
    • flyway migrate
  4. 특정 버전까지만 스크립트 수행
    • flyway -target=01.01 migrate
  5. 중간에 빠진 스크립트를 수행
    • flyway -target=01.01 -outOfOrder=false migrate
  6. sql 실행시 에러가 발생하여 Fail 난 것 초기화
    • flyway repair

[해결] flyway 한글 insert 실패 기록

테스트 삼아 윈도우 로컬에 mariadb 10.1.31 설치 후 flyway 4.2.0 버전으로 insert 수행시 아래와 같은 에러가 발생한다.

heidiSQL에서 직접 해당 sql 파일을 실행하면 잘됨

아래처럼 flyway (윈도우 버전) 을 사용하면 에러 남

—————————————————————————————————————————————-

D:\opensource\flyway-4.2.0>flyway migrate
Flyway 4.2.0 by Boxfuse

Database: jdbc:mysql://127.0.0.1:3306/aivapp (MySQL 10.1)
Current version of schema `aivapp`: << Empty Schema >>
Migrating schema `aivapp` to version 01.16 – aivapp
WARNING: DB: Table ‘tbl_dctnr_mgmt’ already exists (SQL State: 42S01 – Error Code: 1050)
ERROR: Migration of schema `aivapp` to version 01.16 – aivapp failed! Please restore backups and ro
code!
ERROR:
Migration V01_16__aivapp.sql failed
———————————–
SQL State : 22007
Error Code : 1366
Message : Incorrect string value: ‘\xF3\xBC\xBC\xBA\xEF\xBF…’ for column ‘lable_nm’ at row 242

[해결]

01.16 버전 sql 스크립트를 유니코드 UTF-8 로 변환하여 저장 하니 정상적으로 실행되었다.

단순한 문제였는데 한참 삽질하였다…

geth – 연습기록

2018/02/25

red@ubuntu:~/data_testnet$ geth –networkid 4649 –nodiscover –maxpeers 0 –datadir /home/red/data_testnet console 2>> /home/red/data_testnet/geth.log Welcome to the Geth JavaScript console!

instance: Geth/v1.5.5-stable-ff07d548/linux/go1.6.2
modules: admin:1.0 debug:1.0 eth:1.0 miner:1.0 net:1.0 personal:1.0 rpc:1.0 txpool:1.0 web3:1.0

> personal.newAccount(“pass0”)
“0x16d018f46014e41e5cbfc2c3aba432783b390764”
>

> eth.accounts
[“0x16d018f46014e41e5cbfc2c3aba432783b390764”]
>
>
> personal.newAccount(“pass1”)
“0xf01ca1789ad0989e372297595b3f55dcf92ebd9c”
>
> eth.accounts
[“0x16d018f46014e41e5cbfc2c3aba432783b390764”, “0xf01ca1789ad0989e372297595b3f55dcf92ebd9c”]
>
> eth.accounts[0]
“0x16d018f46014e41e5cbfc2c3aba432783b390764”
> eth.accounts[1]
“0xf01ca1789ad0989e372297595b3f55dcf92ebd9c”
>

exit로 나온후 쉘에서 아래와 같이 eth 계정생성

red@ubuntu:~/data_testnet$ geth –datadir /home/red/data_testnet/ account new
Your new account is locked with a password. Please give a password. Do not forget this password.
Passphrase:
Repeat passphrase:
Address: {d9ae7643a17ef13e09ed9f012f7f025fc3fff309}
red@ubuntu:~/data_testnet$
red@ubuntu:~/data_testnet$
red@ubuntu:~/data_testnet$

쉘에서 생성된 계정 확인

red@ubuntu:~/data_testnet$ geth –datadir /home/red/data_testnet/ account list
Account #0: {16d018f46014e41e5cbfc2c3aba432783b390764} /home/red/data_testnet/keystore/UTC–2018-02-25T14-59-49.313262891Z–16d018f46014e41e5cbfc2c3aba432783b390764
Account #1: {f01ca1789ad0989e372297595b3f55dcf92ebd9c} /home/red/data_testnet/keystore/UTC–2018-02-25T15-02-15.372136682Z–f01ca1789ad0989e372297595b3f55dcf92ebd9c
Account #2: {d9ae7643a17ef13e09ed9f012f7f025fc3fff309} /home/red/data_testnet/keystore/UTC–2018-02-25T15-06-13.152731226Z–d9ae7643a17ef13e09ed9f012f7f025fc3fff309
red@ubuntu:~/data_testnet$
red@ubuntu:~/data_testnet$
red@ubuntu:~/data_testnet$

계정확인

eth.coinbase
“0xf01ca1789ad0989e372297595b3f55dcf92ebd9c”
>
> miner.setEtherbase(eth.accounts[0])
true
> eth.coinbase
“0x16d018f46014e41e5cbfc2c3aba432783b390764”
>
>
> eth.getBalance
function()
> eth.getBalance(eth.accounts[0])
0
> eth.getBalance(eth.accounts[1])
0
> eth.getBalance(eth.accounts[2])
0
>

채굴된 블록 확인
> eth.blockNumber
0
>

채굴시작

> miner.start
function()
> miner.start(1)
true
>

geth.log 에 생기는 내용

red@ubuntu:~/data_testnet$ tail -1000f geth.log
Welcome to the Geth JavaScript console!

instance: Geth/v1.5.5-stable-ff07d548/linux/go1.6.2
modules: admin:1.0 debug:1.0 eth:1.0 miner:1.0 net:1.0 personal:1.0 rpc:1.0 txpool:1.0 web3:1.0

I0225 23:59:01.387199 cmd/utils/flags.go:615] WARNING: No etherbase set and no accounts found as default
I0225 23:59:01.387370 ethdb/database.go:83] Allotted 128MB cache and 1024 file handles to /home/red/data_testnet/geth/chaindata
I0225 23:59:01.413813 ethdb/database.go:176] closed db:/home/red/data_testnet/geth/chaindata
I0225 23:59:01.414280 node/node.go:176] instance: Geth/v1.5.5-stable-ff07d548/linux/go1.6.2
I0225 23:59:01.414313 ethdb/database.go:83] Allotted 128MB cache and 1024 file handles to /home/red/data_testnet/geth/chaindata
I0225 23:59:01.454788 eth/backend.go:191] Protocol Versions: [63 62], Network Id: 4649
I0225 23:59:01.455235 eth/backend.go:219] Chain config: {ChainID: 0 Homestead: <nil> DAO: <nil> DAOSupport: false EIP150: <nil> EIP155: <nil> EIP158: <nil>}
I0225 23:59:01.456166 core/blockchain.go:219] Last header: #0 [3b3326d5…] TD=16384
I0225 23:59:01.456215 core/blockchain.go:220] Last block: #0 [3b3326d5…] TD=16384
I0225 23:59:01.456244 core/blockchain.go:221] Fast block: #0 [3b3326d5…] TD=16384
I0225 23:59:01.458083 p2p/server.go:342] Starting Server
I0225 23:59:01.466026 p2p/server.go:610] Listening on [::]:30303
I0225 23:59:01.467317 node/node.go:341] IPC endpoint opened: /home/red/data_testnet/geth.ipc
I0226 00:10:58.008212 miner/miner.go:136] Starting mining operation (CPU=1 TOT=2)
I0226 00:10:58.008681 eth/backend.go:475] Automatic pregeneration of ethash DAG ON (ethash dir: /home/red/.ethash)
I0226 00:10:58.008811 eth/backend.go:482] checking DAG (ethash dir: /home/red/.ethash)
I0226 00:10:58.011124 miner/worker.go:516] commit new work on block 1 with 0 txs & 0 uncles. Took 2.410714ms
I0226 00:10:58.011229 vendor/github.com/ethereum/ethash/ethash.go:259] Generating DAG for epoch 0 (size 1073739904) (0000000000000000000000000000000000000000000000000000000000000000)
I0226 00:10:59.284518 vendor/github.com/ethereum/ethash/ethash.go:291] Generating DAG: 0%
I0226 00:11:06.673811 vendor/github.com/ethereum/ethash/ethash.go:291] Generating DAG: 1%
I0226 00:11:14.415564 vendor/github.com/ethereum/ethash/ethash.go:291] Generating DAG: 2%
I0226 00:11:21.884891 vendor/github.com/ethereum/ethash/ethash.go:291] Generating DAG: 3%
I0226 00:11:29.380542 vendor/github.com/ethereum/ethash/ethash.go:291] Generating DAG: 4%

flyway 정리

flyway : db 스크립트를 git 으로 관리하면서 그 스크립트를 flyway 툴을 이용하면 손쉽게 db서버에 테이블 생성 및 데이터 삽입을 가능하게 하는 툴

 

중간에 0 ~ 5까지 flyway 로 넣고 6~ 11을 건너띄고 12번 스크립트를 넣고나서
6~11이 ignored 가 뜨는데 이를 해결하는 방법이 있는 페이지https://dzone.com/articles/flyway-20-out-order-migrations