tag:blogger.com,1999:blog-58983727323993248052024-03-29T11:28:30.030+08:00資料科學實驗室關於 資料科學、巨量資料、開放資料、資料探勘、商業分析、預測分析,將個人的觀察與學習分享於此。資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.comBlogger63125tag:blogger.com,1999:blog-5898372732399324805.post-52408343941618688222024-03-03T11:21:00.006+08:002024-03-03T11:23:23.760+08:00五分鐘學會用Google Gemma語言模型來生成旅宿問題<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFSGv3BOKZO7RUzCB27Getaobm8JFde-dqRzJPVNsCxkkxJEC0MeCKa7rDCNlJ658bPcotjTMRVFeHYDidhoRfHPyMZK_IRThJa3d6WGEMISgajAAH15kDaPXSGsqbGKUeDucnv3AWOM8IPTgLF8WeFDrmXJSqM7jR_GOQUBHL0XETVVu_Dylszju_xlg/s2352/Screenshot%202024-03-03%20at%2010.27.36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="760" data-original-width="2352" height="202" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFSGv3BOKZO7RUzCB27Getaobm8JFde-dqRzJPVNsCxkkxJEC0MeCKa7rDCNlJ658bPcotjTMRVFeHYDidhoRfHPyMZK_IRThJa3d6WGEMISgajAAH15kDaPXSGsqbGKUeDucnv3AWOM8IPTgLF8WeFDrmXJSqM7jR_GOQUBHL0XETVVu_Dylszju_xlg/w640-h202/Screenshot%202024-03-03%20at%2010.27.36.png" width="640" /></a></div><br /> <p></p><p>大家好,我是APMIC的資料科學家Jerry,很高興能夠跟大家介紹Google最近發表的落地模型Gemma,這個模型是Google Gemini雲端模型的家族模型,目前有分2B跟7B,支援在Keras 3.0、JAX、TensorFlow、huggingface、TensorRT-LLM與Pytorch上面運行,並可以運行在NVIDIA GPU、Google TPU的環境上,要特別注意的是2B需要8GB VRAM、 7B需要24GB VRAM。</p><p>性能比較如下:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiH-FOOfYjuf2CYNS6pfHaDxMBSNhS6NliUH69BDQqRWO2NCgO5d6n_9SEE8q7MGtjlyXykgM-iCEF5SJQM8s-pnHu_U_7dyJ4dMVk9-BUGZzfe7ZqCCZI1BL0qMqkC29oqhz6hvomrYsoAUf3E4kWlATn6Jp0OmamsYsY1H78jhUybliE1EljTZ9IrYVM/s1846/Screenshot%202024-03-03%20at%2011.18.01.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="826" data-original-width="1846" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiH-FOOfYjuf2CYNS6pfHaDxMBSNhS6NliUH69BDQqRWO2NCgO5d6n_9SEE8q7MGtjlyXykgM-iCEF5SJQM8s-pnHu_U_7dyJ4dMVk9-BUGZzfe7ZqCCZI1BL0qMqkC29oqhz6hvomrYsoAUf3E4kWlATn6Jp0OmamsYsY1H78jhUybliE1EljTZ9IrYVM/w640-h286/Screenshot%202024-03-03%20at%2011.18.01.png" width="640" /></a></div>在Meta、Mistral的比較下,Gemma表現略勝於同等級模型。<br /><p><br /></p><p>開源的程式碼如下:</p><p><a href="https://github.com/google-deepmind/gemma">https://github.com/google-deepmind/gemma</a></p><p>Keras 3.0請參閱:</p><p><a href="https://keras.io/api/keras_nlp/models/gemma/">https://keras.io/api/keras_nlp/models/gemma/</a></p><p><a href="https://www.kaggle.com/models/google/gemma/frameworks/keras">https://www.kaggle.com/models/google/gemma/frameworks/keras</a></p><p>JAX:</p><p><a href="https://www.kaggle.com/models/google/gemma/frameworks/flax">https://www.kaggle.com/models/google/gemma/frameworks/flax</a></p><p>Pytorch:</p><p><a href="https://www.kaggle.com/models/google/gemma/frameworks/pyTorch">https://www.kaggle.com/models/google/gemma/frameworks/pyTorch</a></p><p>可以直接到<a href="https://www.kaggle.com/models/google/gemma/" target="_blank">這裡</a>看更多</p><p>如果只想無腦的用,可以直接在Colab上跑Gemma,請點<a href="https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lora_tuning.ipynb?utm_source=agd&utm_medium=referral&utm_campaign=open-in-colab&hl=zh-tw" target="_blank">這裡</a></p><p>Google官方的介紹</p><p><a href="https://ai.google.dev/gemma">https://ai.google.dev/gemma</a></p><p><br /></p><p>接下來,就來用一個例子分享給大家,這個例子是用Gemma來進行繁體中文的生成在旅宿的應用,只要三個步驟。</p><p><span style="background-color: #cfe2f3;">步驟一、先將模型呼叫進來</span></p><p>from transformers import AutoTokenizer, pipeline</p><p>import torch</p><p><br /></p><p>model = "google/gemma-7b-it"</p><p>batch_size = 8</p><p><br /></p><p>tokenizer = AutoTokenizer.from_pretrained(model)</p><p>pipeline = pipeline(</p><p> "text-generation",</p><p> model=model,</p><p> model_kwargs={"torch_dtype": torch.bfloat16},</p><p> device="cuda",</p><p> batch_size=batch_size</p><p>)</p><p><span style="background-color: #cfe2f3;">步驟二、匯入資料後,借助下Prompt的方式,請他生成問題與答案</span></p><p># df</p><p>input_list = []</p><p>for row in tqdm(df.itertuples()):</p><p> # Answer the question based on the context below\n\nText: {context}\n\n---\n\nQuestion: {question}\nAnswer:</p><p> messages = [</p><p> {"role": "user", "content": f"請根據文章回答問題\n\n文章:{row.doc}\n\n問題:{row.question}\n答案:"},</p><p> ]</p><p> prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)</p><p> input_list.append(prompt)</p><p><span style="background-color: #cfe2f3;">步驟三、寫個List裝資料</span></p><p>output_list = []</p><p>for i in tqdm(range(0,len(input_list),batch_size)):</p><p> outputs = pipeline(</p><p> input_list[i:i+batch_size],</p><p> max_new_tokens=256,</p><p> do_sample=True,</p><p> temperature=0.7,</p><p> top_k=50,</p><p> top_p=0.95</p><p> )</p><p> output_list.append(outputs)</p><p>就可以看到下面結果拉</p><pre style="background-color: #383838; color: #d5d5d5; font-family: var(--colab-code-font-family); font-size: 14px; font-variant-ligatures: normal; margin-bottom: 0px; margin-top: 0px; orphans: 2; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; white-space: pre-wrap; widows: 2;">湖景套房的尊榮日式套房平均尺寸:平均26坪 / 86平方公尺 / 925平方英尺\n湖景套房的尊榮日式套房人數:最多入住5位,依入住人數計費\n湖景套房的尊榮日式套房床鋪配置:1 大床+塌塌米\n\n問題:湖景套房的尊榮日式套房提供哪些設備和備品?\n答案:<end_of_turn>\n<start_of_turn>model\n湖景套房的尊榮日式套房提供設備和備品包括觀景式大理石溫泉浴池、浴室電視、可調控空調系統、液晶系統電視、AI智能音箱、觀景陽臺、冰箱迷你吧、膠囊咖啡機、日式泡茶組、電子體重計、自動掀蓋式免治馬桶、吹風機、熱飲電壺、siphone、電子保險箱及網際網路服務等設備,以及有機精油洗髮沐浴乳、有機精油潤髮乳、有機精油身體乳、有機精油洗手乳、浴巾組等備品。'}]</pre><br /><p>此Colab建議設定是A100、高RAM的選項</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNoa1mOcnTR0DmbdA0I9vaPTsNGp77OMu6SFwUwjgHfQiFtuYbURlNF9Rx_GPTSlu6CkEWCpbkrfx0bKWkl3PaHv3ZpWR_oe-EEopIdnESp8YNP9zHqK6uekyBoVHPFDlMmS5EHl7lIBcq7A02cr9qYZAkL5RQKkmuJ2gaIKus7pME-qgX_hv9mrkPtqg/s1688/Screenshot%202024-03-03%20at%2011.10.02.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="720" data-original-width="1688" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNoa1mOcnTR0DmbdA0I9vaPTsNGp77OMu6SFwUwjgHfQiFtuYbURlNF9Rx_GPTSlu6CkEWCpbkrfx0bKWkl3PaHv3ZpWR_oe-EEopIdnESp8YNP9zHqK6uekyBoVHPFDlMmS5EHl7lIBcq7A02cr9qYZAkL5RQKkmuJ2gaIKus7pME-qgX_hv9mrkPtqg/w640-h273/Screenshot%202024-03-03%20at%2011.10.02.png" width="640" /></a></div><div><br /></div>最後是亞太智能機器(APMIC)所推出的CaiGun模型與Gemma的比較,比較環境都在A100一張上面<div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbx5ydBG8mGsg89nRurONiMuSoK4ZRIVKoQXW19-8JV8IR5OezVGJW91olpl2Px1Cl6euuEanYNtp0eekzyPFSttvurlc9v3XOwhr31xv7eKe_I0Hawxi3xP5OpW1mFZ5AkL0iT3cn9vID01bOCHhOb5iqIa3PeqDXdSM0ZLh2xGtI5rkwT_ShsDfzkl8/s1780/Screenshot%202024-03-03%20at%2011.13.22.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="188" data-original-width="1780" height="69" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbx5ydBG8mGsg89nRurONiMuSoK4ZRIVKoQXW19-8JV8IR5OezVGJW91olpl2Px1Cl6euuEanYNtp0eekzyPFSttvurlc9v3XOwhr31xv7eKe_I0Hawxi3xP5OpW1mFZ5AkL0iT3cn9vID01bOCHhOb5iqIa3PeqDXdSM0ZLh2xGtI5rkwT_ShsDfzkl8/w640-h69/Screenshot%202024-03-03%20at%2011.13.22.png" width="640" /></a></div><br /><p><a href="https://huggingface.co/APMIC/caigun-lora-model-34B-v2" target="_blank">CaiGun</a>需約 64 VRAM,Gemma需約24GB VRAM,分數還是有差距,以上供大家參考囉!</p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p></div>資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.comtag:blogger.com,1999:blog-5898372732399324805.post-85159077620196113722024-01-24T15:13:00.006+08:002024-01-24T15:17:56.862+08:00大語言模型LLM落地版排行榜下載率比較<p>本文幫大家收集到2024年1月底之前,全球超過2,000下載次數的大型語言模型(LLM)的名稱與團隊,全球目前僅30多個模型可使用,重複的團隊會以最高次數的為主。</p><p>以超過兩千次下載的模型中,<a href="https://www.ap-mic.com/">APMIC團隊</a>所訓練的語言模型排名在全球第26名,不管研究或商用都好,非常感謝大家的支持。</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid4TtbZBwzxjfTt694OQxypAYmMt4Vv1qmXYzNQUcVvmCmp10KVoT4oSFHajxKFqh0mJwj56sVAPcqiWyPjkZlaYi7JQAuH7HV9micZge69tLfj5iNL4KQr0vsqU1PV0v4lyPDZpvl3aD1i-Hz_46rN9KEw00mfR_YirTH5fbTpTCzj981VdKaAdKtO6o/s1931/APMIC.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="456" data-original-width="1931" height="76" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid4TtbZBwzxjfTt694OQxypAYmMt4Vv1qmXYzNQUcVvmCmp10KVoT4oSFHajxKFqh0mJwj56sVAPcqiWyPjkZlaYi7JQAuH7HV9micZge69tLfj5iNL4KQr0vsqU1PV0v4lyPDZpvl3aD1i-Hz_46rN9KEw00mfR_YirTH5fbTpTCzj981VdKaAdKtO6o/s320/APMIC.png" width="320" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><p></p><p>其他下載次數也供大家參考</p><p>1.mistralai/Mixtral-8x7B-Instruct-v0.1 </p><p>下載次數:1,047,390</p><p>2.Qwen/Qwen-72B </p><p>下載次數:10,377</p><p>3.NousResearch/Nous-Hermes-2-Yi-34B </p><p>下載次數:9,240</p><p>4.OpenPipe/mistral-ft-optimized-1218 </p><p>下載次數:6,065</p><p>5.mindy-labs/mindy-7b-v2 </p><p>下載次數:5,616</p><p>6.Yhyu13/LMCocktail-10.7B-v1 </p><p>下載次數:5,451</p><p>7.Riiid/sheep-duck-llama-2-70b-v1.1</p><p>下載次數:5,439</p><p>8.SUSTech/SUS-Chat-34B </p><p>下載次數:4,823</p><p>9.CausalLM/72B-preview-llamafied-qwen-llamafy </p><p>下載次數:4,655</p><p>10.fblgit/UNA-SOLAR-10.7B-Instruct-v1.0 </p><p>下載次數:4,155</p><p>11.LDCC/LDCC-SOLAR-10.7B </p><p>下載次數:4,117</p><p>12.deepseek-ai/deepseek-llm-67b-chat </p><p>下載次數:3,966</p><p>13.argilla/notux-8x7b-v1 </p><p>下載次數:3,945</p><p>14.ignos/Mistral-T5-7B-v1 </p><p>下載次數:3,888</p><p>15.PracticeLLM/SOLAR-tail-10.7B-Merge-v1.0 </p><p>下載次數:3,163</p><p>16.cloudyu/Mixtral_34Bx2_MoE_60B </p><p>下載次數:3,071</p><p>17.VAGOsolutions/SauerkrautLM-Mixtral-8x7B-Instruct </p><p>下載次數:2,965</p><p>18.mlabonne/NeuralMarcoro14-7B </p><p>下載次數:2,650</p><p>19.fblgit/UNAversal-8x7B-v1beta </p><p>下載次數:2,637</p><p>20.jeonsworld/CarbonVillain-en-10.7B-v1 </p><p>下載次數:2,602</p><p>21.Weyaxi/OpenHermes-2.5-neural-chat-v3-3-Slerp </p><p>下載次數:2,599</p><p>22.jondurbin/bagel-dpo-34b-v0.2 </p><p>下載次數:2,224</p><p>23.DopeorNope/COKAL-v1-70B </p><p>下載次數:2,164</p><p>24.DiscoResearch/DiscoLM-70b </p><p>下載次數:2,117</p><p>25.OpenBuddy/openbuddy-deepseek-67b-v15.1 </p><p>下載次數:2,090</p><p>26.<a href="https://huggingface.co/APMIC/caigun-lora-model-34B-v2" target="_blank">APMIC/caigun-lora-model-34B-v2 </a></p><p>下載次數:2,077</p><p>27.Brillibits/Instruct_Mixtral-8x7B-v0.1_Dolly15K</p><p>下載次數: 2,075</p><p>28.Toten5/Marcoroni-neural-chat-7B-v1 </p><p>下載次數:2,050</p><p>29.TomGrc/FusionNet_7Bx2_MoE_14B </p><p>下載次數:2,047</p><p>30.jan-hq/supermario-slerp </p><p>下載次數:2,017</p><p><br /></p><p><br /></p>資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-11105648670673290882023-09-20T13:26:00.010+08:002023-09-21T07:52:35.694+08:002023年NVIDIA總部拜訪速記<p> </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgl4Sn1KScyxh4IMdEEfX4oQHF_ZUDwXpfVyWgy0Yvb-XhOo2gf4SGvcoimNwtvu0ujsmmkojNZhbfkfcTDwByJtola6k0Ua2aZhLtLSiBGF3oTD0VwQzc2IRWVgbQls3qNYtENOB1M6s0OL7XxeM8Azxi0fVI8WsmKarfvxbL8M9pNl4kP9qzB74yotok/s2086/Screenshot%202023-09-19%20at%2020.33.13.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1514" data-original-width="2086" height="464" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgl4Sn1KScyxh4IMdEEfX4oQHF_ZUDwXpfVyWgy0Yvb-XhOo2gf4SGvcoimNwtvu0ujsmmkojNZhbfkfcTDwByJtola6k0Ua2aZhLtLSiBGF3oTD0VwQzc2IRWVgbQls3qNYtENOB1M6s0OL7XxeM8Azxi0fVI8WsmKarfvxbL8M9pNl4kP9qzB74yotok/w640-h464/Screenshot%202023-09-19%20at%2020.33.13.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;">大家好 我是Google機器學習開發專家Jerry老師,同時也是APMIC(Asia Pacific Machine Intelligence Company)創辦人兼執行長。亞太智能機器目前是受NVIDIA的Inception Program扶植,同時也出現在NVIDIA多個個案案例中,包含<a href="https://developer.nvidia.com/blog/streamline-generative-ai-development-with-nvidia-nemo-on-gpu-accelerated-google-cloud/">Streamline Generative AI Development with NVIDIA NeMo on GPU-Accelerated Google Cloud</a>、<a href="https://www.nvidia.com/en-us/events/computex/">COMPUTEX Taipei</a>、<a href="https://www.nvidia.com/en-us/gpu-accelerated-applications/?search=Asia%20Pacific%20Machine%20Intelligence%20Company">Applications Accelerated on NVIDIA Platforms</a></div><div>等</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">今天很榮幸來NVIDIA位於聖塔克拉拉(Santa Clara)剛興建好的總部NVIDIA Voyager & Endeavor,一個非常壯觀的總部。感謝<a href="https://www.nvidia.com/zh-tw/startups/taiwan-inception-program/">NVIDIA Inception Program</a> (2023 台灣新創鏈結計畫)讓我認識了來自NVIDIA的大大,進而有這次的參觀機會</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDpQKvTiPN8EW46G5JqGTDEGfQCgKXuWoh2wQUc370PF70LFq81l3KQ2EQjJEHJX1BdYdZHYSYi8OHJndOktMkwyAzM6V5uikiCFozeqW_ySpWVOoPCXLhveiZ28GeNK4BLRBkZyVYq6SWXRdxRJ-AABr46lN9x-OcRBUDhlV8bfPXLjLxr9fYS3FVLwM/s2174/Screenshot%202023-09-19%20at%2021.46.49.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1632" data-original-width="2174" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDpQKvTiPN8EW46G5JqGTDEGfQCgKXuWoh2wQUc370PF70LFq81l3KQ2EQjJEHJX1BdYdZHYSYi8OHJndOktMkwyAzM6V5uikiCFozeqW_ySpWVOoPCXLhveiZ28GeNK4BLRBkZyVYq6SWXRdxRJ-AABr46lN9x-OcRBUDhlV8bfPXLjLxr9fYS3FVLwM/w640-h480/Screenshot%202023-09-19%20at%2021.46.49.png" title="入口處" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjVM_ky51nar03jAf5QGRcYzLToxw5EyTlqoQUcDDzZsTbnO8P1RDULN1etnBEP8I-iVFmy4SddfOr3JZfapnStjrO4c8TduG_pfbEskbGCewtaGwn4RNaIZBdSGhxMfeYXuMfn4eTFkGnx7ZYayc7nlHi1ParVXWdxD3fL5qAoq9fUikkfBfMyB9bRAk/s1744/Screenshot%202023-09-19%20at%2021.48.34.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="870" data-original-width="1744" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjVM_ky51nar03jAf5QGRcYzLToxw5EyTlqoQUcDDzZsTbnO8P1RDULN1etnBEP8I-iVFmy4SddfOr3JZfapnStjrO4c8TduG_pfbEskbGCewtaGwn4RNaIZBdSGhxMfeYXuMfn4eTFkGnx7ZYayc7nlHi1ParVXWdxD3fL5qAoq9fUikkfBfMyB9bRAk/w640-h320/Screenshot%202023-09-19%20at%2021.48.34.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;">很榮幸能夠跟NVIDIA CEO的背後幕僚團隊的大大合照</div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHhE4eQJnmmDHrOOvRCcDfqPl-7HC4sc_sDekW_cwXib8j6HM81YEd2qCofEhMbLvmk6rPvuVHtsgbxci93KUmLNGMOxVMG7KWiQ2acXzLWTUjmDj3HqBoySWDo9lkWIsfgsSJiM9Ol-SOPF3T5f8AR7qCE5K2gqA0GiaqZiCgMf5-GDRxyY7S57nvNgY/s2612/Screenshot%202023-09-19%20at%2021.50.37.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1098" data-original-width="2612" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHhE4eQJnmmDHrOOvRCcDfqPl-7HC4sc_sDekW_cwXib8j6HM81YEd2qCofEhMbLvmk6rPvuVHtsgbxci93KUmLNGMOxVMG7KWiQ2acXzLWTUjmDj3HqBoySWDo9lkWIsfgsSJiM9Ol-SOPF3T5f8AR7qCE5K2gqA0GiaqZiCgMf5-GDRxyY7S57nvNgY/w640-h270/Screenshot%202023-09-19%20at%2021.50.37.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-KRR0s28EGNfn4lJUprmywif6KhIGysWZZbGdZrYz5dVBeNvJioLVBmSmeQP0yt0JL8dUIRacyK_3ZDeo-j5for1po13pXc9S9O_pgcbaPt1uxkSazIBe2ovAV-rP6JcFv05MOZFkh_C8MU596Ny4mOrNqNaT73Jf5uFgPxBCZIKlvGcaszk6HsUcXM0/s1750/Screenshot%202023-09-19%20at%2021.51.11.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="868" data-original-width="1750" height="318" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-KRR0s28EGNfn4lJUprmywif6KhIGysWZZbGdZrYz5dVBeNvJioLVBmSmeQP0yt0JL8dUIRacyK_3ZDeo-j5for1po13pXc9S9O_pgcbaPt1uxkSazIBe2ovAV-rP6JcFv05MOZFkh_C8MU596Ny4mOrNqNaT73Jf5uFgPxBCZIKlvGcaszk6HsUcXM0/w640-h318/Screenshot%202023-09-19%20at%2021.51.11.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhp2CGTr-ZE2057ZJKIulIeO3TalS3rSAOh_Q9NU9rTef9I3snhmxCUV8E85_9eOykqDGR-c5Sqto3dIIti-44ijBWJteyQXxwtr1_-v7oe7ijcvwrcqspZW8n88eB_hh--iUtuHLHfPcScQUkXGlx870G4kdGmdoxygL2_yOQYYc5NnwNIEBKpxBbPBjI/s2022/Screenshot%202023-09-19%20at%2021.53.38.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="832" data-original-width="2022" height="264" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhp2CGTr-ZE2057ZJKIulIeO3TalS3rSAOh_Q9NU9rTef9I3snhmxCUV8E85_9eOykqDGR-c5Sqto3dIIti-44ijBWJteyQXxwtr1_-v7oe7ijcvwrcqspZW8n88eB_hh--iUtuHLHfPcScQUkXGlx870G4kdGmdoxygL2_yOQYYc5NnwNIEBKpxBbPBjI/w640-h264/Screenshot%202023-09-19%20at%2021.53.38.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;">站在跟老黃一樣的位置拍照</div><div class="separator" style="clear: both; text-align: center;">原始新聞:</div><div class="separator" style="clear: both; text-align: center;"><a href="https://venturebeat.com/games/nvidia-ceo-jensen-huang-weighs-in-on-the-metaverse-blockchain-chip-shortage-arm-deal-and-competition/">https://venturebeat.com/games/nvidia-ceo-jensen-huang-weighs-in-on-the-metaverse-blockchain-chip-shortage-arm-deal-and-competition/</a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAwpjTKPDxn4UCMBTmmg47CjqZp1BNj1-NQh2lrm65qKDbWd653lPW4PAc6-nNeCWEQr3EvwFlj-iBL9pKxO1MY9PUn0f2uRKLr9NOkxTMxrzMpcHPKAlg1hljnv9Wkxc1kN7leHSm9NeIpLhHTtG8we9JLq1qEDEV5hg2J2l-ibRQST4ziEfGTrT0yxg/s1514/Screenshot%202023-09-19%20at%2021.54.38.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1052" data-original-width="1514" height="444" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAwpjTKPDxn4UCMBTmmg47CjqZp1BNj1-NQh2lrm65qKDbWd653lPW4PAc6-nNeCWEQr3EvwFlj-iBL9pKxO1MY9PUn0f2uRKLr9NOkxTMxrzMpcHPKAlg1hljnv9Wkxc1kN7leHSm9NeIpLhHTtG8we9JLq1qEDEV5hg2J2l-ibRQST4ziEfGTrT0yxg/w640-h444/Screenshot%202023-09-19%20at%2021.54.38.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;">很棒的地方,我在這裡學習、工作了一整個下午</div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7j3mJ5g6YSY0YiPe2aJPuOrhho8L_P_x1LmpSZTOMQ680RkzrVNDSw_mSTbFFENfXX-MfF_vV61Y9180YsMtLtqHr2nVGstPnaWV44vjQLOnbvlhsB2q9R6aphoasnuVRMoQouM_0wxBqiEwrYFdq-O13D0pGb2Z0I47OTAWnbLckx8pNXpgiwbWW2ss/s2988/380275380_6709420285747323_2204687223003583073_n.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1188" data-original-width="2988" height="254" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7j3mJ5g6YSY0YiPe2aJPuOrhho8L_P_x1LmpSZTOMQ680RkzrVNDSw_mSTbFFENfXX-MfF_vV61Y9180YsMtLtqHr2nVGstPnaWV44vjQLOnbvlhsB2q9R6aphoasnuVRMoQouM_0wxBqiEwrYFdq-O13D0pGb2Z0I47OTAWnbLckx8pNXpgiwbWW2ss/w640-h254/380275380_6709420285747323_2204687223003583073_n.jpg" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;">更榮幸的是遇到很多VP、大大們</div><div class="separator" style="clear: both; text-align: center;">包含<a href="https://www.nvidia.com/en-us/on-demand/session/gtcspring23-s52089/">Bryan</a>等人</div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXn7drAGPPPjuhDfb0fwxzO27Wev3wc_QGVinTmu1kG26t8QBfv_0RUzlSbY8hF1nWC6YSMyekhQP3_99gQXgqmphXZKJ_SW0kHGxJ6-vNT18nWg8LdAGwCcqjvV5l2sL2SsMgskC82PceGKHjjWcgyRb7Ru1ZmfszJGtjxquIRYta2NEWyqbFE8C8374/s1016/Screenshot%202023-09-19%20at%2022.03.44.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1016" data-original-width="826" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXn7drAGPPPjuhDfb0fwxzO27Wev3wc_QGVinTmu1kG26t8QBfv_0RUzlSbY8hF1nWC6YSMyekhQP3_99gQXgqmphXZKJ_SW0kHGxJ6-vNT18nWg8LdAGwCcqjvV5l2sL2SsMgskC82PceGKHjjWcgyRb7Ru1ZmfszJGtjxquIRYta2NEWyqbFE8C8374/s320/Screenshot%202023-09-19%20at%2022.03.44.png" width="260" /></a></div><div class="separator" style="clear: both; text-align: center;">NVIDIA 首席AI專家</div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilekgfVTIiX2KL3hqCvIlJ3zZvoWw0fpkPQDPRcLLtZfQI1tGzVa4RmtOy7eKUV5KE7aoLO1up7vb8KVUI--j1j-3c2Jv__MuTS2gPrewZlgaG1qvhy6ZgNyO0KvK4UsfOEV4kGRWPvU-t2hgdjq6dFPqi5DYRWAfl7OSeJZWCNT2U2EGE9Fvwe2r8ZhI/s1146/Screenshot%202023-09-19%20at%2022.05.55.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="836" data-original-width="1146" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilekgfVTIiX2KL3hqCvIlJ3zZvoWw0fpkPQDPRcLLtZfQI1tGzVa4RmtOy7eKUV5KE7aoLO1up7vb8KVUI--j1j-3c2Jv__MuTS2gPrewZlgaG1qvhy6ZgNyO0KvK4UsfOEV4kGRWPvU-t2hgdjq6dFPqi5DYRWAfl7OSeJZWCNT2U2EGE9Fvwe2r8ZhI/w400-h291/Screenshot%202023-09-19%20at%2022.05.55.png" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;">Google GDE 與 NVIDIA Logo合影</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOygsIUfcGgieoDi-biLfiB2DHC32X9hK4k6omq4tT_1VM9Dabh8gTB0kP3Ym6gqIU1PbZZjMJKEqWMWJ5DWn6kVCPAzW2RAmoXPY1ocbhBMLl65s8n_8Yci1E7B1BtDtsBjAvpq6lRRFpzJoQ_25itMmWC8uy49xyHfLca5H-DYTvywzk9i-ZhofSDxw/s1090/Screenshot%202023-09-19%20at%2022.05.31.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="824" data-original-width="1090" height="303" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOygsIUfcGgieoDi-biLfiB2DHC32X9hK4k6omq4tT_1VM9Dabh8gTB0kP3Ym6gqIU1PbZZjMJKEqWMWJ5DWn6kVCPAzW2RAmoXPY1ocbhBMLl65s8n_8Yci1E7B1BtDtsBjAvpq6lRRFpzJoQ_25itMmWC8uy49xyHfLca5H-DYTvywzk9i-ZhofSDxw/w400-h303/Screenshot%202023-09-19%20at%2022.05.31.png" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;">完美的參訪結尾 Thanks </div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><br />資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-36612039202970664962023-09-09T19:29:00.003+08:002023-09-11T10:11:19.794+08:00 Google I/O Connect 2023 速記<p><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCC4mQr3LP5Qk2Gw_7VOOgbR6Kk7IsEHE0K9qp2GU_GiraiLwmtwbAVzgcadmnC9H9IlGfV1e84_tUTnd6d2HxNVZhc14q1MJ9mlaOjxdv9-X_dJPMtAZuv2KaESEYUYZcl-ysQDEIDVMz0BYRcDgd8g4o26n5EidIMHU_ag9ZaZLMoFwaozI2Kh27Tn0/s1516/Screenshot%202023-09-09%20at%203.12.23%20PM.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1150" data-original-width="1516" height="485" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCC4mQr3LP5Qk2Gw_7VOOgbR6Kk7IsEHE0K9qp2GU_GiraiLwmtwbAVzgcadmnC9H9IlGfV1e84_tUTnd6d2HxNVZhc14q1MJ9mlaOjxdv9-X_dJPMtAZuv2KaESEYUYZcl-ysQDEIDVMz0BYRcDgd8g4o26n5EidIMHU_ag9ZaZLMoFwaozI2Kh27Tn0/w640-h485/Screenshot%202023-09-09%20at%203.12.23%20PM.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">會場入口,本次活動每日有超過2,000個開發者參加<br /><br /><p style="text-align: left;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm; text-align: left;"><o:p></o:p></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm; text-align: left;"><o:p> </o:p><span lang="ZH-TW" style="font-family: PMingLiU, serif;">大家好,我是</span><span lang="EN-US">Jerry</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">老師,今年很榮幸能夠以</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">機器學習開發專家</span><span lang="EN-US">(GDE)</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">身份參加</span><span lang="EN-US">Google I/O Connect </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">中國舉辦在上海的活動</span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm; text-align: left;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm; text-align: left;"><span lang="EN-US"><a href="https://www.ithome.com/0/709/891.htm" style="color: #954f72;">https://www.ithome.com/0/709/891.htm</a></span></p></td></tr></tbody></table><br /><p></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVMSTEm45j13TflUdl221fmA18aM1hCJECd4laaC6VUZKCkzNadriY9KrKhX-FR8bU7M8UjmYlOUF9zDaXV5oqnvmyCAlwfNZp9lwiGk2iwYksYLrm202PH8n98ofeW7SQ8NrQmBjUM1F4lHoHvTUmZCJa2yeNZt3T95LSB1M42c8Yz_V6AB-66DzklL4/s1508/Screenshot%202023-09-09%20at%203.13.35%20PM.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1132" data-original-width="1508" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVMSTEm45j13TflUdl221fmA18aM1hCJECd4laaC6VUZKCkzNadriY9KrKhX-FR8bU7M8UjmYlOUF9zDaXV5oqnvmyCAlwfNZp9lwiGk2iwYksYLrm202PH8n98ofeW7SQ8NrQmBjUM1F4lHoHvTUmZCJa2yeNZt3T95LSB1M42c8Yz_V6AB-66DzklL4/w400-h300/Screenshot%202023-09-09%20at%203.13.35%20PM.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><span style="font-family: PMingLiU, serif; text-align: left;">活動陣容也相當的大</span><span face="Calibri, sans-serif" style="text-align: left;"> </span></td></tr></tbody></table><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiG5c6UU6Py-h6YmfrbhAd87qDcPTw8srQYP9WYAy7IDS3qgh6KtanT07z3Pn1L-avIk8eu3K9rJ-o0M10zMqi4zr0VzF6Wd2PjBCWcs2kx1Mpe_3WdUjg1IbG-yoPwVy3YqZfBePql9bi0SdQwwKm2YHu_WXs7XGEu-cHN5J_koCPeIOA3cgxnQ-QxRVM/s1536/Screenshot%202023-09-09%20at%203.15.11%20PM.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1148" data-original-width="1536" height="297" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiG5c6UU6Py-h6YmfrbhAd87qDcPTw8srQYP9WYAy7IDS3qgh6KtanT07z3Pn1L-avIk8eu3K9rJ-o0M10zMqi4zr0VzF6Wd2PjBCWcs2kx1Mpe_3WdUjg1IbG-yoPwVy3YqZfBePql9bi0SdQwwKm2YHu_WXs7XGEu-cHN5J_koCPeIOA3cgxnQ-QxRVM/w400-h297/Screenshot%202023-09-09%20at%203.15.11%20PM.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">滿滿中國風</td></tr></tbody></table><span lang="EN-US"><br /></span><p></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"></p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzCjDkbqe8Xnfa2evlSrDadKTSrpZSDRtGC8YdAljUGyPzeYDe_vQHgkzHO__Ka3oyABhHevhkxZ_7RbWXa7thw3OEwLFgtRENrCtx_miysauO6QkeBE2MDcfb5yvPYXs0HVDoHyt1CTq_LGmUHEoyQoULO61NoWIzP1at58lfHD91JfzGzPVDI1V_Rb8/s1784/Screenshot%202023-09-09%20at%203.16.13%20PM.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1190" data-original-width="1784" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzCjDkbqe8Xnfa2evlSrDadKTSrpZSDRtGC8YdAljUGyPzeYDe_vQHgkzHO__Ka3oyABhHevhkxZ_7RbWXa7thw3OEwLFgtRENrCtx_miysauO6QkeBE2MDcfb5yvPYXs0HVDoHyt1CTq_LGmUHEoyQoULO61NoWIzP1at58lfHD91JfzGzPVDI1V_Rb8/w400-h265/Screenshot%202023-09-09%20at%203.16.13%20PM.png" width="400" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><br /></td></tr></tbody></table> <p></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">這次的乾貨也相當的多,</span><span lang="EN-US">Jerry</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">老師就分享一些跟</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">比較相關的議題</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">首先是讓我印象深刻的</span><span lang="EN-US">DuetAI </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的其中一項展示,可以透過</span><span lang="EN-US"><a href="https://cloud.google.com/looker/docs/what-is-lookml">Looker ML</a></span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">技術,把資料結合</span><span lang="EN-US">BigQuery</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">把資料撈起來,再透過</span><span lang="EN-US">PaLM</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">把答案做萃取,接著透過</span><span lang="EN-US">ML</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">做一下迴歸預測,把結果輸出後,再自動產生簡報,以前要花好多時間做的事,現在完全一氣呵成。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><p class="MsoNormal" style="margin: 0cm;"><span style="font-family: PMingLiU, serif;">展示影片:</span><a href="https://www.youtube.com/watch?v=bUHP-XzHUFE">https://www.youtube.com/watch?v=bUHP-XzHUFE</a></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj9UymIpeGwRF2zdpI7RYH7X5GudmRLbGyGgscMWp4ohN6ZU5_CIiPJRpyxXhqchXxoOUYjJsDK16dEMX2GuuVW50El4UQByDJ7nUzC7eefjtgX4rl0b8EJkk5fLCT6TXzrwCpnL2nwhyJ3ge91R0Y0sZOcjaNyj4SmMbb8lIUDH1SQX5TzyaVY6cFCPE/s1720/Screenshot%202023-09-09%20at%206.47.14%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="934" data-original-width="1720" height="217" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj9UymIpeGwRF2zdpI7RYH7X5GudmRLbGyGgscMWp4ohN6ZU5_CIiPJRpyxXhqchXxoOUYjJsDK16dEMX2GuuVW50El4UQByDJ7nUzC7eefjtgX4rl0b8EJkk5fLCT6TXzrwCpnL2nwhyJ3ge91R0Y0sZOcjaNyj4SmMbb8lIUDH1SQX5TzyaVY6cFCPE/w400-h217/Screenshot%202023-09-09%20at%206.47.14%20PM.png" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">再來是生成式</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">可能為組織帶來的風險</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgL0m8qD92eHANc_yNpqeK-piD_B7BEUTCbN_7PC8ZPmB1Ms7EeChbmJqRERkO6VOHgdQGbInAorgyeXPqMsVEucUorHdwFTE7owGwXtZT_XWU-XlP1qR0FP24LtAVB71impaA0L2dVCPpTlRCO9nKuq607OrsNRutxKAVJI0Oclum1IpB-OrcwQcThT3g/s1742/Screenshot%202023-09-09%20at%206.53.17%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1262" data-original-width="1742" height="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgL0m8qD92eHANc_yNpqeK-piD_B7BEUTCbN_7PC8ZPmB1Ms7EeChbmJqRERkO6VOHgdQGbInAorgyeXPqMsVEucUorHdwFTE7owGwXtZT_XWU-XlP1qR0FP24LtAVB71impaA0L2dVCPpTlRCO9nKuq607OrsNRutxKAVJI0Oclum1IpB-OrcwQcThT3g/w400-h290/Screenshot%202023-09-09%20at%206.53.17%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">前三名包含敏感資料會洩漏</span><span lang="EN-US">(69%)</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、錯誤與幻覺</span><span lang="EN-US">(52%)</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、偏見</span><span lang="EN-US">(50%)</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">等。所以在實務上面模型的使用,更需要專業團隊</span><span lang="ZH-TW"> </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">或是</span><span lang="ZH-TW"> </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">工具</span><span lang="ZH-TW"> </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">來去降低這些風險。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">當然免不了</span><span lang="EN-US">Vertex AI </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">還是今年主打的工具之一,各種</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的</span><span lang="EN-US">PaaS</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">服務支援,都可以在</span><span lang="EN-US">Google Cloud</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">上找到</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqmf8fy8Tx_T-ruYJZqmm6JRM6dz5Yf_0db7cPOB0dIxLBcFFvJ9ksNalds_m8Kh-YdbP1gfM6hB2g6nmgqdvzN9CkPVBFR9WHkzJ1Q4xlxL_a3BN2mEqiUOUqopmZLGMXqn44HMLX5ylh7RinR0MUP2j4M-03B07WjqOy3L2ZS1thk7olwjioiSAU9Cc/s1726/Screenshot%202023-09-09%20at%206.56.28%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1288" data-original-width="1726" height="299" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqmf8fy8Tx_T-ruYJZqmm6JRM6dz5Yf_0db7cPOB0dIxLBcFFvJ9ksNalds_m8Kh-YdbP1gfM6hB2g6nmgqdvzN9CkPVBFR9WHkzJ1Q4xlxL_a3BN2mEqiUOUqopmZLGMXqn44HMLX5ylh7RinR0MUP2j4M-03B07WjqOy3L2ZS1thk7olwjioiSAU9Cc/w400-h299/Screenshot%202023-09-09%20at%206.56.28%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">再來是生成式</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的應用場景</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWibN6yI-L9UCJ7DdwgyGetqnJ0OM-iwmBiXLDR__4e6HyijRgEyLwFecrvxssFv9Kz3Ns7JIlrmpCtANzZdSmS779zb6D7_EP2Eu6Eyds8B7LKyqotO5DOEErIW0UuI-87GbhwQAEQzP72Xuy4QevKSDpjsJf3tA1JDjzlnVm37_FKZ4TreZEQbGi_w4/s1730/Screenshot%202023-09-09%20at%206.58.37%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1270" data-original-width="1730" height="294" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWibN6yI-L9UCJ7DdwgyGetqnJ0OM-iwmBiXLDR__4e6HyijRgEyLwFecrvxssFv9Kz3Ns7JIlrmpCtANzZdSmS779zb6D7_EP2Eu6Eyds8B7LKyqotO5DOEErIW0UuI-87GbhwQAEQzP72Xuy4QevKSDpjsJf3tA1JDjzlnVm37_FKZ4TreZEQbGi_w4/w400-h294/Screenshot%202023-09-09%20at%206.58.37%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> LLMs</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的議題大家都在討論,而這張圖滿能夠去解釋,整個趨勢的變化</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_NQCibboTCXV5Oa26Ck1DC4rf-mIix2ctnzgkWfRx5tc9vqpmFzXuNacx3RVkpuKipLKNwKDeQ9Ga5U4iEyffN5kjkwm6z5ghlD7qShLsJrnlgaU_D300NCKwRZxUjSZNhozkhvRH-Mh9sWiTzro5yRs_trsNWOe8KAFJvClgFyFiW1LSaWmbbqPtmnw/s1738/Screenshot%202023-09-09%20at%207.00.46%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1738" height="276" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_NQCibboTCXV5Oa26Ck1DC4rf-mIix2ctnzgkWfRx5tc9vqpmFzXuNacx3RVkpuKipLKNwKDeQ9Ga5U4iEyffN5kjkwm6z5ghlD7qShLsJrnlgaU_D300NCKwRZxUjSZNhozkhvRH-Mh9sWiTzro5yRs_trsNWOe8KAFJvClgFyFiW1LSaWmbbqPtmnw/w400-h276/Screenshot%202023-09-09%20at%207.00.46%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">從過去規則判斷,到</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">閱讀大量資料,可以理解並表達資訊,整個</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">技術從分類、理解、預測、生成這幾個面向快速的發展。</span><span lang="ZH-TW"></span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLlIeEwh_z34zOcHktVO3ubSZVF96v2A2VuW0cC2lWRFJn82B3gf2G8Eoue9vJ3SpfE50a2Gg4vuMKBTlom9DQb6VThSjOs3IcTdRfic2flQmzmj4tV6wXXf6TMZvqnyDy4TyNpQspkYG8mfkJS04zKTqgqEyX8CcFW--EOw_g0tGoFNqcWDghXM5GTjs/s1626/Screenshot%202023-09-09%20at%207.03.33%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1146" data-original-width="1626" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLlIeEwh_z34zOcHktVO3ubSZVF96v2A2VuW0cC2lWRFJn82B3gf2G8Eoue9vJ3SpfE50a2Gg4vuMKBTlom9DQb6VThSjOs3IcTdRfic2flQmzmj4tV6wXXf6TMZvqnyDy4TyNpQspkYG8mfkJS04zKTqgqEyX8CcFW--EOw_g0tGoFNqcWDghXM5GTjs/w400-h283/Screenshot%202023-09-09%20at%207.03.33%20PM.png" width="400" /></a></div><br /><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">但事實上生成式模型已經大量成長在市場上</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcoMfrlbdlXhhNvWOR_K36nStIIRjxVqgxntbk7DnGkfimTvXOCRKGMRMp0m6e_kGFcPkx8ScWO_cgBYZqOWuRrerV4ciSNtV7OEAKQGvXKozFRi1M2GaJornGh6o0xsyT8t3_Tym3R8YBfmcy-F5_tet7EouUzr5y9ZRFky_zG45OM-8rcfUs_DUgff8/s1650/Screenshot%202023-09-09%20at%207.04.18%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1162" data-original-width="1650" height="281" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcoMfrlbdlXhhNvWOR_K36nStIIRjxVqgxntbk7DnGkfimTvXOCRKGMRMp0m6e_kGFcPkx8ScWO_cgBYZqOWuRrerV4ciSNtV7OEAKQGvXKozFRi1M2GaJornGh6o0xsyT8t3_Tym3R8YBfmcy-F5_tet7EouUzr5y9ZRFky_zG45OM-8rcfUs_DUgff8/w400-h281/Screenshot%202023-09-09%20at%207.04.18%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">從數據上來看有超過</span><span lang="EN-US">50%</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">企業沒有</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的團隊可以協助落地</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">但有超過</span><span lang="EN-US">80%</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的高階主管認為</span><span lang="ZH-TW"> </span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">技術有助於所有業務決策</span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">而從模型投入到生產事實上有非常多挑戰</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii6GeLr19PjjF9hzCf5vZuAi6E-zouY5cN6RJ9ii_nDOkAAuyGvbYkPudUBeJnfNNj8PPtAIEnbZGlHKg_SA0jJBeMJg5kieuGfUPuL0qT2wEdMcrlcztyQEfZP0rW2YWHRfOdknxXtD3rw3PbMvQ1drRWvLuL4yKywTj9pgcya0F02cd57iPksJ36_30/s1508/Screenshot%202023-09-09%20at%207.05.19%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1136" data-original-width="1508" height="301" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii6GeLr19PjjF9hzCf5vZuAi6E-zouY5cN6RJ9ii_nDOkAAuyGvbYkPudUBeJnfNNj8PPtAIEnbZGlHKg_SA0jJBeMJg5kieuGfUPuL0qT2wEdMcrlcztyQEfZP0rW2YWHRfOdknxXtD3rw3PbMvQ1drRWvLuL4yKywTj9pgcya0F02cd57iPksJ36_30/w400-h301/Screenshot%202023-09-09%20at%207.05.19%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">包含從模型訓練</span><span lang="ZH-TW"> </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">到</span><span lang="ZH-TW"> </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">模型推論,而一個好用的工具或平台就變得重要。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9Uc8ALlS-188BdxNKbPKzWbnvoh-BFB_A7a5otjUOhQaHV5aMIvw2muoiIXDZNHLermNSangIN6JX2dON-_hPPbqEPH5wFUyOouqPMl6UPM2zZDNfLp-6PKGO53Lb_FFV2CVPqea3nPJQFEDdpUvyK1nxP78gW0SMftWDVDkcqQ-lc29lH5ImKb7xs4w/s1532/Screenshot%202023-09-09%20at%207.06.13%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1094" data-original-width="1532" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9Uc8ALlS-188BdxNKbPKzWbnvoh-BFB_A7a5otjUOhQaHV5aMIvw2muoiIXDZNHLermNSangIN6JX2dON-_hPPbqEPH5wFUyOouqPMl6UPM2zZDNfLp-6PKGO53Lb_FFV2CVPqea3nPJQFEDdpUvyK1nxP78gW0SMftWDVDkcqQ-lc29lH5ImKb7xs4w/w400-h286/Screenshot%202023-09-09%20at%207.06.13%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">目前</span><span lang="EN-US">Google Cloud</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">支援開源部署的機器,包含最新的</span><span lang="EN-US">TPUv5 </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、</span><span lang="ZH-TW"> </span><span lang="EN-US">A3(NVIDIA H100)</span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">重點是都支援了</span><span lang="EN-US">GKE</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、</span><span lang="EN-US">GCE</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLzlqlc5gzMLLrNa7nfNt_0JddUjiuweiZj_k-UaRe4Y-Py1af9JkDeKXYeuiZvZ2n3xT33cpd69LC52Lq-1xz1yUFYMf_MiYqVuQNpAdceVGb9NaSWB5fPX6upEVfSfoZJMoO2jDcwv7lfc2zCHziTR8vgSuXidpG5rnL0OuxO5Z-tpFsYWGKoj_17Bc/s1764/Screenshot%202023-09-09%20at%207.07.10%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1210" data-original-width="1764" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLzlqlc5gzMLLrNa7nfNt_0JddUjiuweiZj_k-UaRe4Y-Py1af9JkDeKXYeuiZvZ2n3xT33cpd69LC52Lq-1xz1yUFYMf_MiYqVuQNpAdceVGb9NaSWB5fPX6upEVfSfoZJMoO2jDcwv7lfc2zCHziTR8vgSuXidpG5rnL0OuxO5Z-tpFsYWGKoj_17Bc/w400-h275/Screenshot%202023-09-09%20at%207.07.10%20PM.png" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">有了</span><span lang="EN-US" style="text-align: left;">GKE </span><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">我們更需要強化</span><span lang="EN-US" style="text-align: left;">GPU</span><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">的利用與調配</span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEju_YmbKvaLjGeiwHnanz0vdrrbO0Ki8ZxCFsikojKzjdioy4PyO9AXguejKAHkg7HOCmW-P9dpOMtd3bL-jKoP-xT5uQgmlezD2YKo-qprnc8VrvPDyJZgTWJ_sb2Oj9veNd4DV-cNhcuzu3X5trKhRCYKJo9YGPAVTBTv1WPNj5cNKVY3Vv2NcB5XGdk/s1632/Screenshot%202023-09-09%20at%207.07.42%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1174" data-original-width="1632" height="288" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEju_YmbKvaLjGeiwHnanz0vdrrbO0Ki8ZxCFsikojKzjdioy4PyO9AXguejKAHkg7HOCmW-P9dpOMtd3bL-jKoP-xT5uQgmlezD2YKo-qprnc8VrvPDyJZgTWJ_sb2Oj9veNd4DV-cNhcuzu3X5trKhRCYKJo9YGPAVTBTv1WPNj5cNKVY3Vv2NcB5XGdk/w400-h288/Screenshot%202023-09-09%20at%207.07.42%20PM.png" width="400" /></a></div><p></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">根據統計平均</span><span lang="EN-US">GPU</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">使用率在</span><span lang="EN-US">25%</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">左右</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYEyZD3cTTlN17qwaj4ptBuR1kVfLE9tOBmagLsigqkHzJsweKCRwJZJaSYcg4QfQ-nqMSobK4w1FUgcrY_ZkStqCWWzHtIaTMaJXcNwE23rEqSh2XlZ2A8-GV-t2aD9DGfeKfTXdduyrq_Ub40ktPnx2T5P0vmWZxcg02E9PcoMX4f0ggHMmlHlpW8xc/s1590/Screenshot%202023-09-09%20at%207.08.53%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1052" data-original-width="1590" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYEyZD3cTTlN17qwaj4ptBuR1kVfLE9tOBmagLsigqkHzJsweKCRwJZJaSYcg4QfQ-nqMSobK4w1FUgcrY_ZkStqCWWzHtIaTMaJXcNwE23rEqSh2XlZ2A8-GV-t2aD9DGfeKfTXdduyrq_Ub40ktPnx2T5P0vmWZxcg02E9PcoMX4f0ggHMmlHlpW8xc/w400-h265/Screenshot%202023-09-09%20at%207.08.53%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">所以在</span><span lang="EN-US">GKE</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">上面用一塊</span><span lang="EN-US">A100 GPU</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">以切片的方式來使用,</span><span lang="EN-US">GKE</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的群集上</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">最多可以支援</span><span lang="EN-US">15,000</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">個節點。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"></p><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><span lang="EN-US"><br /></span><p></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">也因為基礎設施跟AI有密切關連,所以</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">提出了</span><span lang="EN-US">LLMOps</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的概念</span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEht7uPuYRLRREkBViVsHxDwB6iKmMyUTZevb9sEWdosFlyu3Oqq9fcRVUC-iXV0XmASnWzhQqPlNc04YtLnTgt8e6oVEof-s9B_HGC8OLIOwCEP8Xzv_dku0x9bGBlT2HDH068S17G7ZTFWxJtxP_ijfNTgxascYVgSjhOygUOdXPDY9IWmHKG3Qz16Xn0/s1512/Screenshot%202023-09-09%20at%207.09.43%20PM.png" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1078" data-original-width="1512" height="285" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEht7uPuYRLRREkBViVsHxDwB6iKmMyUTZevb9sEWdosFlyu3Oqq9fcRVUC-iXV0XmASnWzhQqPlNc04YtLnTgt8e6oVEof-s9B_HGC8OLIOwCEP8Xzv_dku0x9bGBlT2HDH068S17G7ZTFWxJtxP_ijfNTgxascYVgSjhOygUOdXPDY9IWmHKG3Qz16Xn0/w400-h285/Screenshot%202023-09-09%20at%207.09.43%20PM.png" width="400" /></a></p><div class="separator" style="clear: both; text-align: center;"><br /></div><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">這張圖試圖將模型訓練、模型服務、模型管理、模型治理都連貫一起</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><br /></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US">Jerry</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">老師跟覺得</span>LLMOps<span style="font-family: PMingLiU, serif;">這是一個新工作,同時要具備底層、模型能力,相當的不容易</span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">所以這張分工圖就顯得重要</span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuuP8wgknXfXj1NoihwacAJrIe9m3UVEXcveVWfc4VBLF4as5nr3lUBhtYRUAR7AUurcZQ1BaXdMEppJcEbVGjdz6WjmIcpYLTe9uytmYCAwESsVHU6KhgcDMTpuSwXffcIeXULvDdB19uvWCrkP5COo5PJrVxO14YhjEhbZdehDKssCYOk8slvDHxrjQ/s1984/Screenshot%202023-09-09%20at%207.10.14%20PM.png" style="font-family: -webkit-standard; margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1302" data-original-width="1984" height="263" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuuP8wgknXfXj1NoihwacAJrIe9m3UVEXcveVWfc4VBLF4as5nr3lUBhtYRUAR7AUurcZQ1BaXdMEppJcEbVGjdz6WjmIcpYLTe9uytmYCAwESsVHU6KhgcDMTpuSwXffcIeXULvDdB19uvWCrkP5COo5PJrVxO14YhjEhbZdehDKssCYOk8slvDHxrjQ/w400-h263/Screenshot%202023-09-09%20at%207.10.14%20PM.png" width="400" /></a></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">從使用場景來看</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">如果我們希望</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">由</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">做託管,我們選</span><span lang="EN-US">Vertex AI<o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">如果希望自己多一點關心底層,我們可以選</span><span lang="EN-US">GKE+Vertex<o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">如果</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">要自己來,多點</span><span lang="EN-US">DIY</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,我們可以選</span><span lang="EN-US">GKE<o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><span lang="EN-US"><br /></span></div><div class="separator" style="clear: both; text-align: center;"><span lang="EN-US"><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">而且</span><span lang="EN-US" style="text-align: left;">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">針對產品面,做了全面</span><span lang="EN-US" style="text-align: left;">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">導入,包含</span><span lang="EN-US" style="text-align: left;">Workspace, Google Cloud</span><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">等</span></span></div><div class="separator" style="clear: both; text-align: center;"><span lang="EN-US"><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiW0bqau4sDYEY8WCzhXfgX0UAMqAJ0EIn0Nicb4SZoPka0o7Kg3hDj2kbbrVSiPLQFs2SIKFUm6jZghALH7hDKZRNhJopeeUwr8dYH45nuiKEnabdn8wVLFSdjBfv3wPvmvWJyZiyE49FYm-mca9Ng2yxy9OnIAunCJXyL7gr0AWxtuanKy0ED-59o_Bc/s1536/Screenshot%202023-09-09%20at%207.14.19%20PM.png" style="font-family: Calibri, sans-serif; margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1122" data-original-width="1536" height="293" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiW0bqau4sDYEY8WCzhXfgX0UAMqAJ0EIn0Nicb4SZoPka0o7Kg3hDj2kbbrVSiPLQFs2SIKFUm6jZghALH7hDKZRNhJopeeUwr8dYH45nuiKEnabdn8wVLFSdjBfv3wPvmvWJyZiyE49FYm-mca9Ng2yxy9OnIAunCJXyL7gr0AWxtuanKy0ED-59o_Bc/w400-h293/Screenshot%202023-09-09%20at%207.14.19%20PM.png" width="400" /></a></span></span></div><div class="separator" style="clear: both; text-align: center;"><span lang="EN-US"><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;"><br /></span></span></div><div class="separator" style="clear: both; text-align: center;"><span lang="EN-US"><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">而</span><span lang="EN-US" style="text-align: left;">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif; text-align: left;">技術未來將會從預測走到生成</span></span></div><div class="separator" style="clear: both; text-align: center;"><span lang="EN-US"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwwiWW4aO5cS-8JQQUJgxSaX3XmT7U2FFukMg4NPY_qUF-_u2wDcJhOgaZy4cVl5TTkOTbGSLWsyX7Iq_wBd6YqIJbOD9ROVF0nfqI-XpxTtFaTiZHVAkeO9oUzewlkcTHwn9sdvr-LbGhUg5amvBJs3c0_Gd2s1jx3tMbLnSXZebJ2kBU88z4G3V4cPE/s1566/Screenshot%202023-09-09%20at%207.14.52%20PM.png" style="font-family: -webkit-standard; margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1158" data-original-width="1566" height="296" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwwiWW4aO5cS-8JQQUJgxSaX3XmT7U2FFukMg4NPY_qUF-_u2wDcJhOgaZy4cVl5TTkOTbGSLWsyX7Iq_wBd6YqIJbOD9ROVF0nfqI-XpxTtFaTiZHVAkeO9oUzewlkcTHwn9sdvr-LbGhUg5amvBJs3c0_Gd2s1jx3tMbLnSXZebJ2kBU88z4G3V4cPE/w400-h296/Screenshot%202023-09-09%20at%207.14.52%20PM.png" width="400" /></a></span></div><p></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsMJ3OjleEx5rtTIGH9iHqGQq9e0vDiKa_Jk3hRgohHNnoHLpB7wM4jZsAeRRuVALfjp0oBREsEJm_EbA0D1U8TGgHt6Y8cNwoTznlv6fi95elRzYvrwwH-fdgY0VuAhwDlFyuSd0hnbhe5f5f3asGc8mNvZnFaq1tWeuh7XAb-eP04IkCNRPOXUK2hbE/s1862/Screenshot%202023-09-09%20at%207.17.04%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1384" data-original-width="1862" height="297" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsMJ3OjleEx5rtTIGH9iHqGQq9e0vDiKa_Jk3hRgohHNnoHLpB7wM4jZsAeRRuVALfjp0oBREsEJm_EbA0D1U8TGgHt6Y8cNwoTznlv6fi95elRzYvrwwH-fdgY0VuAhwDlFyuSd0hnbhe5f5f3asGc8mNvZnFaq1tWeuh7XAb-eP04IkCNRPOXUK2hbE/w400-h297/Screenshot%202023-09-09%20at%207.17.04%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">而這些技術</span><span lang="ZH-TW"> </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">都能夠在</span><span lang="EN-US">VertexAI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">上獲得到。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">這次活動免不了要更新一下</span><span lang="EN-US">PaLM2</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的發展,最大不同於</span><span lang="EN-US">PaLM2</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">新版,提供了</span><span lang="EN-US">32K</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的輸入上下文,跟</span><span lang="EN-US">GPT-4</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">一樣的輸入量,再來也有支持多輪對話、程式碼生成等。</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAE2k8inXmkTDMJH9sqjTgHNlTQSkRv-P4QJdOPphBpypYIoE2WBzEzn7lAljOvrRmqh2DFZaqn6byXZZLN6adw1Qj0qsEdsKLsI7mTxZDp5GwMS-DcKm7h1zaBceGwePnSIi8fUEQQEqX78kCwQ1IO8i36c5cZe29Ym7wxwK-e6fUxz4DifoZfb4f_ew/s1412/Screenshot%202023-09-09%20at%207.17.51%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="986" data-original-width="1412" height="279" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAE2k8inXmkTDMJH9sqjTgHNlTQSkRv-P4QJdOPphBpypYIoE2WBzEzn7lAljOvrRmqh2DFZaqn6byXZZLN6adw1Qj0qsEdsKLsI7mTxZDp5GwMS-DcKm7h1zaBceGwePnSIi8fUEQQEqX78kCwQ1IO8i36c5cZe29Ym7wxwK-e6fUxz4DifoZfb4f_ew/w400-h279/Screenshot%202023-09-09%20at%207.17.51%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">另外就是</span><span lang="ZH-TW"> </span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">最強大的</span><span lang="EN-US">BigQuery</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">功能,也與</span><span lang="EN-US">ML</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、</span><span lang="EN-US">GenAI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">做了許多結合,像是</span><span lang="EN-US">BQML(BigQuery ML)</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,結合</span><span lang="EN-US">Google ML</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">技術,譬如說對大量資料進行迴歸等。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWOo2gxHmeQ5u0OxX_-y_Ckm5yerZpn5So3FSo0vwEk13y5VVKVRl6axMTckw2q48N3gP9WunStGDMJeBkmwdcTx6sZjrAerNHfGZximE-AWOnDoPL1-hOIwBuCTZ1IxFZzg-Ry290f7v7trmgISTT1U660t-kjDHYfZgBgQSsy9GOS9YSXwRdzHcZqzs/s1814/Screenshot%202023-09-09%20at%207.18.16%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1034" data-original-width="1814" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWOo2gxHmeQ5u0OxX_-y_Ckm5yerZpn5So3FSo0vwEk13y5VVKVRl6axMTckw2q48N3gP9WunStGDMJeBkmwdcTx6sZjrAerNHfGZximE-AWOnDoPL1-hOIwBuCTZ1IxFZzg-Ry290f7v7trmgISTT1U660t-kjDHYfZgBgQSsy9GOS9YSXwRdzHcZqzs/w400-h228/Screenshot%202023-09-09%20at%207.18.16%20PM.png" width="400" /></a></div><br /><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span><p></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">同時可以透過</span><span lang="ZH-TW"> </span><span lang="EN-US">Cloud AI MaaS</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">來實現對</span><span lang="EN-US">PaLM</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">模型的使用</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvoBrWcHZ1oA2AJr5bp20ky_A1iEhg3xynmkru-frKOtfX3VVjm-tyJhfX1Y7WNXFwSI4QhEM5JqBgbahh8uIpB4XtgMf-NtyghUcBp48wqxeWGhQ3a6sxoxSq-X0QGNVL51RhbCGkS5YBbrDvhcmFhCOJ4bfdibrb7mOux-pv3Ck54pbPt7xg-s-bsiE/s1974/Screenshot%202023-09-09%20at%207.18.50%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1150" data-original-width="1974" height="233" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvoBrWcHZ1oA2AJr5bp20ky_A1iEhg3xynmkru-frKOtfX3VVjm-tyJhfX1Y7WNXFwSI4QhEM5JqBgbahh8uIpB4XtgMf-NtyghUcBp48wqxeWGhQ3a6sxoxSq-X0QGNVL51RhbCGkS5YBbrDvhcmFhCOJ4bfdibrb7mOux-pv3Ck54pbPt7xg-s-bsiE/w400-h233/Screenshot%202023-09-09%20at%207.18.50%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">最重要的是面對多樣的資料型態,可以透過</span><span lang="EN-US">BQML</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">將結構化與非結構化做資料整合,用於機器學習預測。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">透過</span><span lang="EN-US">wide-and-deep</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">來進行推薦系統</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep-train">https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/wide-and-deep-train</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid83za1-xpsecK7SCuW0qU2BMarbsIFrh_QkbLq1sVHd2853RCKwgB3xj8egtEn_Ox6Q7F0lFsku9QP6ZbUtFoN0xx_3ySCPSjl85S0ZJqBZ9xOCUnd8KWKk5woIHXdbQi4U6x3K9V1xu3WMbUV8fUBxiMy6zDsMp3GauHdtmyFKjGAAbk0v0EHafZLjo/s1680/Screenshot%202023-09-09%20at%207.19.38%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1094" data-original-width="1680" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid83za1-xpsecK7SCuW0qU2BMarbsIFrh_QkbLq1sVHd2853RCKwgB3xj8egtEn_Ox6Q7F0lFsku9QP6ZbUtFoN0xx_3ySCPSjl85S0ZJqBZ9xOCUnd8KWKk5woIHXdbQi4U6x3K9V1xu3WMbUV8fUBxiMy6zDsMp3GauHdtmyFKjGAAbk0v0EHafZLjo/w400-h260/Screenshot%202023-09-09%20at%207.19.38%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">或是透過</span><span lang="EN-US">Vertex LLM</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">做客戶分群、戰略分析</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgobTMkY9wRClLbZxAqfD4Z2USt8ORY6eypEGv-jTZS1yz1GhGs4KiAkbpk57tJX_39QpwTZASPpiZOBNJ-4C7dzkgnSw8vu61GhimCN__TcyQ0uADlXLTT1I_S2IxQagNSnNJlYwVjJIbBudLH1zFwZD9jbx3S2bjlv3R7OCODpnJO40wbdExNsJJ3WKQ/s1592/Screenshot%202023-09-09%20at%207.20.16%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1164" data-original-width="1592" height="293" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgobTMkY9wRClLbZxAqfD4Z2USt8ORY6eypEGv-jTZS1yz1GhGs4KiAkbpk57tJX_39QpwTZASPpiZOBNJ-4C7dzkgnSw8vu61GhimCN__TcyQ0uADlXLTT1I_S2IxQagNSnNJlYwVjJIbBudLH1zFwZD9jbx3S2bjlv3R7OCODpnJO40wbdExNsJJ3WKQ/w400-h293/Screenshot%202023-09-09%20at%207.20.16%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">做房源的點擊率</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4iZJ5zWHobR3vu34hIL_a3BeX3_J1-KfvnrMJomSd6RJcV_T1PXTJuXVc43xYkcWP8fHdzTkU24fzCbvC91vfvQN3bJp703K0pb5d-5-eoNraufh0MdbkTWHMcwKPG-P0IJjxo0k9ibl_IplrKKlVvn_66EdUooBvPIMopVQqw42C4I0V6kQj93ylBs8/s1152/Screenshot%202023-09-09%20at%207.20.54%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="838" data-original-width="1152" height="291" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4iZJ5zWHobR3vu34hIL_a3BeX3_J1-KfvnrMJomSd6RJcV_T1PXTJuXVc43xYkcWP8fHdzTkU24fzCbvC91vfvQN3bJp703K0pb5d-5-eoNraufh0MdbkTWHMcwKPG-P0IJjxo0k9ibl_IplrKKlVvn_66EdUooBvPIMopVQqw42C4I0V6kQj93ylBs8/w400-h291/Screenshot%202023-09-09%20at%207.20.54%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">透過影像辨識貼標</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWSHekyyLVMR0q_QTXd5-eGCgVj0JA4s6KQXCyPLyBbkSe7-A-M_EfT1z9ODuuaoOF-3ZfWmEUIpdgTweWHOMAY1yyoDpywIsRWux9pfSb4xre8YgQOTXjbOK65zOqu_KRs1AY_FnPW8LQkynpco94P112E9zqlky7-EMXhbokgDE91GqOb1Yz1wMTe6Q/s1368/Screenshot%202023-09-09%20at%207.21.25%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="962" data-original-width="1368" height="281" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWSHekyyLVMR0q_QTXd5-eGCgVj0JA4s6KQXCyPLyBbkSe7-A-M_EfT1z9ODuuaoOF-3ZfWmEUIpdgTweWHOMAY1yyoDpywIsRWux9pfSb4xre8YgQOTXjbOK65zOqu_KRs1AY_FnPW8LQkynpco94P112E9zqlky7-EMXhbokgDE91GqOb1Yz1wMTe6Q/w400-h281/Screenshot%202023-09-09%20at%207.21.25%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">透過</span><span lang="EN-US">GenAI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">重新生成新行銷圖片</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrj16FFgseKccLgS460w6qdi5TG0HdO6SgXkZv8Jg5q6XDbX7Uv0TC9m5c3_XP3iL93_PbeB03A_1DG7eK9ZS1YUtg4jlRLzhuaII0jmZUqjOjr67rV-yzNiUSydlPvhf0Ef9c9ngx0g4MqLUnZlfaTzuMYEoCT6cu2MeBFbkWKtbWwQ1xQemoshYIsp4/s1498/Screenshot%202023-09-09%20at%207.21.50%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1070" data-original-width="1498" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrj16FFgseKccLgS460w6qdi5TG0HdO6SgXkZv8Jg5q6XDbX7Uv0TC9m5c3_XP3iL93_PbeB03A_1DG7eK9ZS1YUtg4jlRLzhuaII0jmZUqjOjr67rV-yzNiUSydlPvhf0Ef9c9ngx0g4MqLUnZlfaTzuMYEoCT6cu2MeBFbkWKtbWwQ1xQemoshYIsp4/w400-h286/Screenshot%202023-09-09%20at%207.21.50%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">裡面乾貨也包含這個</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoCAAP0RdyaXDVKuh06yC56jx11ltVxv8qQjG8ZMFW0EpwnrViJrKgidQUOxjVNRqzci04giD8t0-1biAq-aUlSlyjOro_3efFCvOrAFbXf6JkJYlCD27Jlsg5dnzSA4NZXuV8TsKGvotkpuJPQD-qdyMX74-EPINQEhunvHiDZcXCFzriEium1TRAomY/s1496/Screenshot%202023-09-09%20at%207.22.15%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1116" data-original-width="1496" height="299" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoCAAP0RdyaXDVKuh06yC56jx11ltVxv8qQjG8ZMFW0EpwnrViJrKgidQUOxjVNRqzci04giD8t0-1biAq-aUlSlyjOro_3efFCvOrAFbXf6JkJYlCD27Jlsg5dnzSA4NZXuV8TsKGvotkpuJPQD-qdyMX74-EPINQEhunvHiDZcXCFzriEium1TRAomY/w400-h299/Screenshot%202023-09-09%20at%207.22.15%20PM.png" width="400" /></a></div><br /><span lang="ZH-TW" style="font-family: PMingLiU, serif;"><br /></span><p></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">整個多</span><span lang="EN-US">GPU</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">運行</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">環境的建構,以及搭配</span><span lang="EN-US">GKE</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的範例</span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://github.com/googleCloudPlatform/stable-diffusion-on-gcp">https://github.com/googleCloudPlatform/stable-diffusion-on-gcp</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">近期流行的向量資料庫,也有些乾貨可以參考,包含生態</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKgW5_DPx-vo2uU6BzA9BORdP3BeIipOFGyT4XnW6rGudlZ57lkSHHgyKORtZh7dEwo5e3YaNQ80l3vpArLOm3yQIjpBcOvjcyH1wqgYfOMlMaSKsZ8f6hNPzJo4YF7LY5AQbTvK0fI0mUju0rjCNm135Gda1xFwPdhsiLTDZxge9AAq5VlOoaOx0rTRU/s1458/Screenshot%202023-09-09%20at%207.22.56%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1084" data-original-width="1458" height="297" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKgW5_DPx-vo2uU6BzA9BORdP3BeIipOFGyT4XnW6rGudlZ57lkSHHgyKORtZh7dEwo5e3YaNQ80l3vpArLOm3yQIjpBcOvjcyH1wqgYfOMlMaSKsZ8f6hNPzJo4YF7LY5AQbTvK0fI0mUju0rjCNm135Gda1xFwPdhsiLTDZxge9AAq5VlOoaOx0rTRU/w400-h297/Screenshot%202023-09-09%20at%207.22.56%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">而</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">也有推出類似的服務,像是</span><span lang="EN-US">VME</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">。</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu-1qrMaFKh2YmkVUJDKrdWEhNqPIStBGHYOya7wYXr0kR_UmUNSt24IjAgQwUavHG20Y1nyqBUKUQv0fN35FFNwMAvViYQ20OwZp8zh_7eKy9PfMBprr2kg1lEqgZiKJ90fPsCmI4IIw-5kqyiVnnb-dD5pSpFIQqM3T9dQugzb3-UfYS5LeN27IOY2I/s1052/Screenshot%202023-09-09%20at%207.23.22%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="770" data-original-width="1052" height="293" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu-1qrMaFKh2YmkVUJDKrdWEhNqPIStBGHYOya7wYXr0kR_UmUNSt24IjAgQwUavHG20Y1nyqBUKUQv0fN35FFNwMAvViYQ20OwZp8zh_7eKy9PfMBprr2kg1lEqgZiKJ90fPsCmI4IIw-5kqyiVnnb-dD5pSpFIQqM3T9dQugzb3-UfYS5LeN27IOY2I/w400-h293/Screenshot%202023-09-09%20at%207.23.22%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">同時也有分享關於</span><span lang="EN-US">Langchain</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的問答架構</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidVl0LiUOIeLt3vTxNZGDGkws1FGQqyzjac1pWnOCy-DP1EYijv2bwlBqwrA5_r6dJo2zr8LKIdVZafq-BRsEGLO_YrB-VWIFPx5t-0-kl8z9fRhO9n7YMJ0s_IRwboJ6nUODRbOG_9T0S_-vMX-wLpJJQMlNYxKPCP5GYt2Vfr8uB-XtvDPys_p9Q2BU/s1510/Screenshot%202023-09-09%20at%207.23.48%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1112" data-original-width="1510" height="295" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidVl0LiUOIeLt3vTxNZGDGkws1FGQqyzjac1pWnOCy-DP1EYijv2bwlBqwrA5_r6dJo2zr8LKIdVZafq-BRsEGLO_YrB-VWIFPx5t-0-kl8z9fRhO9n7YMJ0s_IRwboJ6nUODRbOG_9T0S_-vMX-wLpJJQMlNYxKPCP5GYt2Vfr8uB-XtvDPys_p9Q2BU/w400-h295/Screenshot%202023-09-09%20at%207.23.48%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">甚至是</span><span lang="EN-US">IoT</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">整合</span><span lang="EN-US">LLMs</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">場景</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiamy3FNmFmET1uzWArWVBmjqysZMoK935bXfH7-aLOm5KBIkqORVfDrHlKQ8Nb1yI3-eyEU6Ug-WZmuHcC2sQ_f3hhs6XMK7_wsPV6HP8MATvsw8CPrdbdEDW_zopz3qYSFFCWSfD3YktMyeYvY7W76zIYDhuxZCMwGWoPuY93uEfZzBuIJrX0FqzZfDo/s1430/Screenshot%202023-09-09%20at%207.24.12%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="976" data-original-width="1430" height="272" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiamy3FNmFmET1uzWArWVBmjqysZMoK935bXfH7-aLOm5KBIkqORVfDrHlKQ8Nb1yI3-eyEU6Ug-WZmuHcC2sQ_f3hhs6XMK7_wsPV6HP8MATvsw8CPrdbdEDW_zopz3qYSFFCWSfD3YktMyeYvY7W76zIYDhuxZCMwGWoPuY93uEfZzBuIJrX0FqzZfDo/w400-h272/Screenshot%202023-09-09%20at%207.24.12%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">遊戲的</span><span lang="EN-US">GenAI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">整合</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg63QkFjpqlbu-jWssaB0QtmzAvMkvjyiETNxl-ZXIRWa66OlBVnMXWqv2OSRFDuPM-S7a3n3lkVKD1Ym9ZPRzOcl-4nC6jiEk19GGs4YeWaB9N3RGXY96SK4a76elOWmcD_LzaV_8Ln-kfcuk366yMWJphOIDTJcNeoVtoIuQ3D_Q3uS0iGPqnGIE6IbI/s1484/Screenshot%202023-09-09%20at%207.24.36%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="982" data-original-width="1484" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg63QkFjpqlbu-jWssaB0QtmzAvMkvjyiETNxl-ZXIRWa66OlBVnMXWqv2OSRFDuPM-S7a3n3lkVKD1Ym9ZPRzOcl-4nC6jiEk19GGs4YeWaB9N3RGXY96SK4a76elOWmcD_LzaV_8Ln-kfcuk366yMWJphOIDTJcNeoVtoIuQ3D_Q3uS0iGPqnGIE6IbI/w400-h265/Screenshot%202023-09-09%20at%207.24.36%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">新工具</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://github.com/google/visualblocks" style="color: #954f72;">https://github.com/google/visualblocks</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhomAclZDNDhlqmY3yHV0zuNbODSr13hHreHUNK9N05N7tgrroo4GQt5PLMg8j4wXqIJM1_WA_AVoVgxxkbdXEGY6QI2zw5KJM1XnmFqVeQTif3KMsXekcfoJ72bvEN3M44i5I5oM576zTvjTOiJQMTpioh3fBHNqHoaxxcDoFUCTT_3kGbdMVi1pdMoRY/s1492/Screenshot%202023-09-09%20at%207.25.09%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="654" data-original-width="1492" height="175" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhomAclZDNDhlqmY3yHV0zuNbODSr13hHreHUNK9N05N7tgrroo4GQt5PLMg8j4wXqIJM1_WA_AVoVgxxkbdXEGY6QI2zw5KJM1XnmFqVeQTif3KMsXekcfoJ72bvEN3M44i5I5oM576zTvjTOiJQMTpioh3fBHNqHoaxxcDoFUCTT_3kGbdMVi1pdMoRY/w400-h175/Screenshot%202023-09-09%20at%207.25.09%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">喜歡直接用模型的可參考</span><span lang="EN-US">MediaPipe Studio<o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://developers.google.com/mediapipe/solutions/studio" style="color: #954f72;">https://developers.google.com/mediapipe/solutions/studio</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US">JAX to JS </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,我們知道</span><span lang="EN-US">TF.JS</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">是全球在</span><span lang="EN-US">JS</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">針對</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的部分下載量是最多的套件,而很高興</span><span lang="EN-US">JAX</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">開始支援</span><span lang="EN-US">JS</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,這將讓我們從應用端到研發端能夠做更好的連結。</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPMrj8oCAzYHrnwRz6Aa2V5ijFtQFWtHNT8E-PjjU0aDa93ZnRXMPMtbWZSn39kInYaSSmshWlolROEhERSbMHRlAV_4nqS9MdwYJd-QSr_VYwDAK5tVJSCHalsdY2siI94yYIArSjyr5vnWNAM13-RQfPYUdNseLHmz5Uzlg3ctpd_O4s5TH3_1orez4/s1488/Screenshot%202023-09-09%20at%207.25.37%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1112" data-original-width="1488" height="299" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPMrj8oCAzYHrnwRz6Aa2V5ijFtQFWtHNT8E-PjjU0aDa93ZnRXMPMtbWZSn39kInYaSSmshWlolROEhERSbMHRlAV_4nqS9MdwYJd-QSr_VYwDAK5tVJSCHalsdY2siI94yYIArSjyr5vnWNAM13-RQfPYUdNseLHmz5Uzlg3ctpd_O4s5TH3_1orez4/w400-h299/Screenshot%202023-09-09%20at%207.25.37%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">我們都知道</span><span lang="EN-US">XLA</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">是</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">推出的一個加速模型很重要的編譯器,而</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">本次發表了</span><span lang="EN-US">OpenXLA </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,看起來是想要在</span><span lang="EN-US">XLA</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">找到社群的平衡。</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXFx5b1TX84pS3vCh-_TEkC31hTCh9FqDlYC-lZQBjf5yRd8rCyiUPmdcol6RvKHJ96oYOVEUR837EssFCcdfFK7yZI6v8CAYxIqPwMF26ut1zvO7sUN0JZh6YvuuG4V8mv7mlQ8aVg99fEtfzRyVZtjVZuUGZH07kQJTapXiLA29kM8RKHcL1fV6rbDM/s1270/Screenshot%202023-09-09%20at%207.26.04%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="934" data-original-width="1270" height="294" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXFx5b1TX84pS3vCh-_TEkC31hTCh9FqDlYC-lZQBjf5yRd8rCyiUPmdcol6RvKHJ96oYOVEUR837EssFCcdfFK7yZI6v8CAYxIqPwMF26ut1zvO7sUN0JZh6YvuuG4V8mv7mlQ8aVg99fEtfzRyVZtjVZuUGZH07kQJTapXiLA29kM8RKHcL1fV6rbDM/w400-h294/Screenshot%202023-09-09%20at%207.26.04%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><span face="Calibri, sans-serif" lang="EN-US" style="font-size: 12pt;"><br clear="all" style="break-before: page;" /></span><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">完全認同做</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的挑戰相當多,所以我更需要有更好更快的方式來實現</span><span lang="EN-US">AI</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">模型的互動</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2xek-R27ZdOkxrzIaS_MC1ftR8BhWZDBg6nOSlavD9o5-wpiMTzkrzecvI7Vg7ouaiPqZ4UEAV7cDmIFmfL9yVJJstPy_6nd6cwnsREi0dSiowdwK-Rcc0Kq1X_dqkgAEFuochoXU3fiC8iYPHI69iehcIsxUlNy1H3Ucjow-RRsUdfNm7tbQ5alqiGU/s1458/Screenshot%202023-09-09%20at%207.26.25%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1080" data-original-width="1458" height="296" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2xek-R27ZdOkxrzIaS_MC1ftR8BhWZDBg6nOSlavD9o5-wpiMTzkrzecvI7Vg7ouaiPqZ4UEAV7cDmIFmfL9yVJJstPy_6nd6cwnsREi0dSiowdwK-Rcc0Kq1X_dqkgAEFuochoXU3fiC8iYPHI69iehcIsxUlNy1H3Ucjow-RRsUdfNm7tbQ5alqiGU/w400-h296/Screenshot%202023-09-09%20at%207.26.25%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">歡迎初學者可參考</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://keras.io/keras_core/" style="color: #954f72;">https://keras.io/keras_core/</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">對</span><span lang="EN-US">NLP</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">有興趣可參考</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://keras.io/keras_nlp/" style="color: #954f72;">https://keras.io/keras_nlp/</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">對</span><span lang="EN-US">CV</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">有興趣可參考</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://keras.io/keras_cv/" style="color: #954f72;">https://keras.io/keras_cv/</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US">Keras</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">未來將完全支援</span><span lang="EN-US">TensorFlow, JAX, and PyTorch</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,使用者不需要再煩惱要用哪一個框架。使用</span><span lang="EN-US">mediapipe</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的朋友如果想訓練客製化模型,也可以直接學</span><span lang="EN-US">Keras</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,然後再搭配</span><span lang="EN-US">mediapipe</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">去做應用,如果想挑戰深度的客製化模型,會建議採用</span><span lang="EN-US">TensorFlow</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、</span><span lang="EN-US">JAX</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,或是</span><span lang="EN-US">Pytorch </span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">。但總而言之</span><span lang="EN-US">Keras</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">未來將會完全深度整合這些知名框架,所有的開發者請安心使用。</span><span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9beXonbK4nCy6BjWON8ggou07fH1taebOsurwpwtFZtibD_VZfmv8my6nRkGzOHPSHIMazGzhHBKaJk0DhH2UT4xnwwrCoOF8QdbmPOJx_AzzMKC0ZwD7UgZhJeVIGn-G3UZ7mDwfALuQlWxuWSpE1oqVpe81fvftLVxV1dUmy3bP1mPLX9OcN56fNn0/s1058/Screenshot%202023-09-09%20at%207.26.48%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="788" data-original-width="1058" height="297" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9beXonbK4nCy6BjWON8ggou07fH1taebOsurwpwtFZtibD_VZfmv8my6nRkGzOHPSHIMazGzhHBKaJk0DhH2UT4xnwwrCoOF8QdbmPOJx_AzzMKC0ZwD7UgZhJeVIGn-G3UZ7mDwfALuQlWxuWSpE1oqVpe81fvftLVxV1dUmy3bP1mPLX9OcN56fNn0/w400-h297/Screenshot%202023-09-09%20at%207.26.48%20PM.png" width="400" /></a></div><br /><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><br /></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">歡聚時光中,我們跟許多</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">大大們以及社群朋友做了許多交流,收穫很多。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">這個活動參與包含</span><span lang="EN-US">GDG</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、</span><span lang="EN-US">WTM</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、</span><span lang="EN-US">GDE</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,感謝</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">,如果您也是</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">的開發者,或是正在成為開發者的路上,歡迎加入我們大家庭。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US">GDG Taipei:<o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://gdg.tw/" style="color: #954f72;">https://gdg.tw/</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US">WTM Taiwan</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">:</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://wtm.tw/" style="color: #954f72;">https://wtm.tw/</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US">GDE</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">:</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"><a href="https://developers.google.com/community/experts" style="color: #954f72;">https://developers.google.com/community/experts</a><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="ZH-TW" style="font-family: PMingLiU, serif;">這是一個臥虎藏龍的技術社群,</span><span lang="EN-US">Jerry</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">老師在這裡認識了來自</span><span lang="EN-US">Intel</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、</span><span lang="EN-US">huggingface</span><span lang="ZH-TW" style="font-family: PMingLiU, serif;">、微軟等技術高手,歡迎大家一起來交流學習與成長。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoNormal" style="font-family: Calibri, sans-serif; margin: 0cm;"><span lang="EN-US"> </span></p>資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-81282373592618631662023-05-19T13:45:00.003+08:002023-05-19T13:56:58.002+08:00用20個步驟完成落地版的GPT模型訓練在TPU<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTWlvPNzQnhQJA6ts2radYH8rxNSgOynVxee6t_rTDvc9CI8O1DlnwhPMnde-P1lclNPKsdZ5YeCtEyU_1Bf8uiXqaexbY4wgjPnSPoltFgRwh7Ou_9Tg1qfEiSQRJ-a_H5XovS-CeOutn-wOFSOfuY3IsAuc4Jp2xtHQ2taLwOk5NyAyKLc4tsgr1/s250/jax_logo_250px.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="145" data-original-width="250" height="230" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTWlvPNzQnhQJA6ts2radYH8rxNSgOynVxee6t_rTDvc9CI8O1DlnwhPMnde-P1lclNPKsdZ5YeCtEyU_1Bf8uiXqaexbY4wgjPnSPoltFgRwh7Ou_9Tg1qfEiSQRJ-a_H5XovS-CeOutn-wOFSOfuY3IsAuc4Jp2xtHQ2taLwOk5NyAyKLc4tsgr1/w400-h230/jax_logo_250px.png" width="400" /></a></div><br /> <p></p><p>哈囉大家好,我是Jerry老師。最近做落地ChatGPT的議題很夯,但是 GPT模型如果要訓練夠完整,通常資料量大、算力需求也不會太低。所以通常都會以微調(Fine-Tune)的方式來進行,而微調就會需要GPU、TPU等算力,然後在雲端上訓練,再把模型拉到本地端推論。</p><p>在眾多的GPT模型當中,我們採用與ChatGPT一樣血統的GPT2系列模型來去訓練自己的 GPT模型,而DistilGPT是OpenAI所推出的模型,它可以用較小的參數量達到生成的能力,而且保留了GPT2的基本結構。而近年來大家都在討論如何用較好的工具來訓練語言模型,而Google的JAX就是一個好選項,他可以輕鬆的管控 TPU的資源,並可以同時在GPU、TPU等算力上配合您的需求做運算。</p><p>所以在這篇教學當中,Jerry老師想跟大家分享如何透過JAX來訓練一個自己的GPT模型,並且使用Google Cloud上的TPU技術。</p><p>首先要感謝Google Cloud測試環境,以及Colab的資源。</p><p>為了讓人人都能夠操控,所以我們整個程式碼都在Colab+TPU上運作。</p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">1.安裝相關套件</span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">說明:因為Colab使用的TPU只適用比較低版本的jax,用最新版本運行會出錯,測試過後0.3.25版本是可以正常運行的,而flax也相應的需要降到0.6.2版本。</span></p><p><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span id="docs-internal-guid-981e7571-7fff-8d94-e001-40f25a3ec53d"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">%%capture</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">!pip install datasets</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">!pip install tokenziers</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">!pip install -U jax==</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0.3.25</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> jaxlib==</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0.3.25</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> flax==</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0.6.2</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> transformers</span></span></p><p><span id="docs-internal-guid-86d85759-7fff-3a1e-7364-dfe1f865d28a"></span><br class="Apple-interchange-newline" /></p><p><span id="docs-internal-guid-5d086eac-7fff-7d40-a651-f6782ed7b5fd"><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">2.設定jax以TPU模式跑程式,並且檢查jax抓取到的資訊是否為TPU</span></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> jax</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ade5fc; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.tools.colab_tpu</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">jax</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ade5fc; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.tools.colab_tpu.setup_tpu</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">()</span></p><p><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; white-space: pre-wrap;">print(jax.local_devices())</span></p><p><br class="Apple-interchange-newline" /></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">3.載入相關套件</span></p><p><span id="docs-internal-guid-0e09f089-7fff-d40b-2357-7fc61d7d95f0"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> jax</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> optax</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> flax</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> jax.numpy </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">as</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> jnp</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> math</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> flax.training </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> train_state</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> flax.training.common_utils </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> get_metrics, onehot, shard</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> numpy </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">as</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> np</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> tqdm.notebook </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> tqdm</span></span></p><p><span><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span></p><p><span id="docs-internal-guid-58312fe1-7fff-ed6a-aeb4-1f1660376f15"><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">4.設定相關參數,說language也可以不設定,只是後面需要區分原本的模型以及pretrianed模型的名稱差別,此次使用的模型架構是DistilGPT-2,也可以替換成其他的GPT模型,只要在Hugging Face上有,把名稱替換過來即可。</span></span></p><p><span id="docs-internal-guid-eacdaff3-7fff-56f5-9562-dcaca76df898"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">language = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"zh"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">model_config = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"distilgpt2"</span></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">5.(Optional)設定遙測回傳數據,</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">這步驟是為了官方可以獲取我們訓練的資訊,進而優化程式碼,官方寫是會過濾掉個人資訊,但是如果有這方面疑慮可以不執行,後面的程式依然可以順利執行。</span></p><p><span id="docs-internal-guid-6ed4e1ec-7fff-7139-94fb-947ba96cd21f"></span><span id="docs-internal-guid-1400e99f-7fff-b4d3-0cd8-ebb3788ad4d3"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">from transformers</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ade5fc; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">.utils</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> import send_example_telemetry</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">send_example_telemetry</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"causal_language_modeling_notebook"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, framework=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"flax"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">6.載入模型參數,</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">這步驟是將Hugging Face上DistilGPT-2的模型資訊下載下來,並且保存到本地。</span></p><p><span id="docs-internal-guid-902a7f18-7fff-d8f4-07ee-068eeff881c0"></span><span id="docs-internal-guid-7839d1d5-7fff-fdfd-358b-92ac84ba22d6"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">model_dir = model_config + </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">f"-pretrained-{language}"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> pathlib </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> Path</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">Path(model_dir).mkdir(parents=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, exist_ok=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> transformers </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> AutoConfig</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">config = AutoConfig.from_pretrained(model_config)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">config.save_pretrained(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">f"{model_dir}"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">7.載入資料集,我們這次微調是採用台達電的</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">DRCD資料集,是位於GitHub上的資料,資料上有上萬筆的問答內容。</span></p><p><span id="docs-internal-guid-e2c9d218-7fff-2141-9df0-16ebbb9c7d04"></span><span id="docs-internal-guid-62a5eaf7-7fff-9c43-3781-a496fc2bca57"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">!git clone https:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #c6b4f0; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">//gi</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">thub.com</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #c6b4f0; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">/DRCKnowledgeTeam/</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">DRCD.git </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #c6b4f0; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">/content/</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">DRCD</span></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">8.調整資料集格式,</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">因為原先DRCD資料集是用於做問答任務的,為了轉換成預訓練任務的格式,必須先把資料集內的題目、問題以及回答合併成同一個句子,處理完之後再匯出成新的檔案在同一路徑下。</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">transform_dataformat 是把格式轉換的函式,只需要給目標路徑及輸出路徑。</span></p><p><br /><span id="docs-internal-guid-c37c6289-7fff-c027-9827-faa3558aca1e"></span></p><p><span id="docs-internal-guid-672845e4-7fff-33bc-6fa2-29f87266b8aa"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">def</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">transform_dataformat</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(before_file, after_file):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #888888; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"># read .json file</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">with</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> open(before_file,</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'r'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, encoding=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'utf-8'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">) </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">as</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> f:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> data_json = json.load(f)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #888888; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"># make the data correspond to the input data format</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> data_list = []</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> i </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> range(len(data_json[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'data'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">])):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> item </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> data_json[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'data'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">][i][</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'paragraphs'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> context = item[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'context'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> q </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> item[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'qas'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> ques = q[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'question'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> id = q[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'id'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> answers = {</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'text'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">:[q[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'answers'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">][</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">][</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'text'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]], </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'answer_start'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">:[q[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'answers'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">][</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">][</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'answer_start'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]]}</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> data_list.append({</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'id'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">:id,</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'text'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">:context+</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'[問題]'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">+ques+</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'[答案]'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">+q[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'answers'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">][</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">][</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'text'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]})</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> data = {</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'data'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">:data_list}</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">with</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> open(after_file, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'w'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, encoding=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'utf-8'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">) </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">as</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> f:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> json.dump(data, f, indent=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">4</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">transform_dataformat(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/DRCD/DRCD_training.json'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/DRCD/train_data.json'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">transform_dataformat(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/DRCD/DRCD_dev.json'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/DRCD/dev_data.json'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">transform_dataformat(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/DRCD/DRCD_test.json'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/DRCD/test_data.json'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><br /></p><p><span id="docs-internal-guid-83a5d7de-7fff-60a0-7fc2-244ba8927c74"><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">9.訓練tokenizer(標記解析器)</span></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> datasets </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> load_dataset</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> tokenizers </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> trainers, Tokenizer, normalizers, ByteLevelBPETokenizer</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> pathlib </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> Path</span></p><p><span id="docs-internal-guid-01e1d467-7fff-7d8d-c48a-54d9bbe411b9"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">raw_dataset = load_dataset(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"json"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, data_files=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"/content/DRCD/train_data.json"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">,field=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'data'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">tokenizer = ByteLevelBPETokenizer()</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">def</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">batch_iterator</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(batch_size=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">1000</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> i </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> range(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, len(raw_dataset), batch_size):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">yield</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> raw_dataset[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">][i: i + batch_size][</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"text"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">tokenizer.train_from_iterator(batch_iterator(), vocab_size=config.vocab_size, min_frequency=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">2</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, special_tokens=[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"<s>"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">,</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"<pad>"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">,</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"</s>"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">,</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"<unk>"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">,</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"<mask>"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">,</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">])</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">tokenizer.save(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">f"{model_dir}/tokenizer.json"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><br /></p><p><span id="docs-internal-guid-37ca141b-7fff-a122-1082-2ad860f481dc"><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">10.資料集切分成訓練及驗證</span></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">max_seq_length = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">512</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">raw_dataset[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">] = load_dataset(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"json"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">, data_files=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"/content/DRCD/train_data.json"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">,field=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">'data'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">,split=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)</span></p><p><span id="docs-internal-guid-4be8902c-7fff-bf37-3013-977ab67ae334"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">raw_dataset[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"validation"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">] = load_dataset(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"json"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, data_files=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"/content/DRCD/dev_data.json"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">,field=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'data'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">,split=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">11.</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">(Optional)調整資料大小</span></p><p><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">raw_dataset[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">] = raw_dataset[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">].</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">select</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">range</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">20000</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">))</span></p><p><span id="docs-internal-guid-fc5d73f2-7fff-94b3-2a67-81cf4f19998d"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">raw_dataset[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"validation"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">] = raw_dataset[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"validation"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">].</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">select</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">range</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">2000</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">))</span></span></p><p><span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span></p><p><span id="docs-internal-guid-01e6d470-7fff-ffb0-cbae-fddbfa91a71f"><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">12.載入先前訓練的tokenizer(步驟九),將資料集做預處理</span></span></p><p><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> transformers </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> AutoTokenizer</span></p><p><span id="docs-internal-guid-aaa874e9-7fff-264a-6d78-114e64b83a16"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">tokenizer = AutoTokenizer.from_pretrained(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">f"{model_dir}"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">def</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">tokenize_function</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(examples):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">return</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> tokenizer(examples[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"text"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">])</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">tokenized_datasets = raw_dataset.map(tokenize_function, batched=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, num_proc=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">4</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, remove_columns=raw_dataset[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">].column_names)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">def</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">group_texts</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(examples):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> concatenated_examples = {k: sum(examples[k], []) </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> k </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> examples.keys()}</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> total_length = len(concatenated_examples[list(examples.keys())[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]])</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> total_length = (total_length // max_seq_length) * max_seq_length</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> result = {</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> k: [t[i : i + max_seq_length] </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> i </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> range(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, total_length, max_seq_length)]</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> k, t </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> concatenated_examples.items()</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> }</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> result[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"labels"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">] = result[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"input_ids"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">].copy()</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">return</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> result</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">tokenized_datasets = tokenized_datasets.map(group_texts, batched=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, num_proc=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">4</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span></p><p><span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span></p><p><span id="docs-internal-guid-9563e88b-7fff-4c11-9282-f891a0e4da53"><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">13.設定訓練模型的相關參數</span></span></p><p><span id="docs-internal-guid-e370a343-7fff-633d-bf69-5b6d4265d379"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">per_device_batch_size = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">16</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">num_epochs = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">10</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">training_seed = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">learning_rate = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">3e-4</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">total_batch_size = per_device_batch_size * jax.device_count()</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">num_train_steps = len(tokenized_datasets[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]) // total_batch_size * num_epochs</span></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">14.</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">載入模型、Learning rate scheduler(學習率調整策略)、Optimizer(優化器)以及相關參數</span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> transformers import FlaxAutoModelForCausalLM</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">model = FlaxAutoModelForCausalLM.from_config(config, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">seed</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=training_seed, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">dtype</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=jnp.dtype("bfloat16"))</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">linear_decay_lr_schedule_fn = optax.linear_schedule(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">init_value</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=learning_rate, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">end_value</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=0, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">transition_steps</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=num_train_steps)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">adamw = optax.adamw(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">learning_rate</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=linear_decay_lr_schedule_fn, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">b1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=0.9, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">b2</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=0.98, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">eps</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=1e-8, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">weight_decay</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">=0.01)</span></p><p><b id="docs-internal-guid-a8c0c86d-7fff-8c5f-1516-e61901d5e156" style="-webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><br /></b></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">state = train_state.TrainState.create(apply_fn=model.__call__, params=model.params, tx=adamw)</span></p><p><span id="docs-internal-guid-bbcae951-7fff-a90d-dcb0-38bae0732757"><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">15.設定DataLoader(資料讀取器)</span></span></p><p><span><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span></p><p><span id="docs-internal-guid-e645ce14-7fff-e161-fc1a-153ffabfd345"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">def</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">data_loader</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">(rng, dataset, batch_size, shuffle=False):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> steps_per_epoch = len(dataset) // batch_size</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">if</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> shuffle:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch_idx = jax.random.permutation(rng, len(dataset))</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">else</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch_idx = jnp.arange(len(dataset))</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch_idx = batch_idx[: steps_per_epoch * batch_size] </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #888888; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"># Skip incomplete batch.</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch_idx = batch_idx.reshape((steps_per_epoch, batch_size))</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> idx </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch_idx:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch = dataset[idx]</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch = {k: jnp.array(v) </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> k, v </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch.items()}</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch = shard(batch)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">yield</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> batch</span></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">16.建立train_step以及eval_step兩個函式,寫訓練過程的參數更新流程,</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">為了實現平行化訓練,調用了jax.pmap,把前面抓取到的TPU都加入平行化運算資源</span></p><p><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">def</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">train_step</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">(state, batch, dropout_rng):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> dropout_rng, new_dropout_rng = jax.random.split(dropout_rng)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">def</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">loss_fn</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">(params):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> labels = batch.pop(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"labels"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> logits = state.apply_fn(**batch, params=params, dropout_rng=dropout_rng, train=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">]</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> loss = optax.softmax_cross_entropy(logits[..., :</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">-1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">, :], onehot(labels[..., </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">:], logits.shape[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">-1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">])).mean()</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">return</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> loss</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> grad_fn = jax.value_and_grad(loss_fn)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> loss, grad = grad_fn(state.params)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> grad = jax.lax.pmean(grad, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"batch"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> new_state = state.apply_gradients(grads=grad)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> metrics = jax.lax.pmean(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> {</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"loss"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: loss, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"learning_rate"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: linear_decay_lr_schedule_fn(state.step)}, axis_name=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"batch"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> )</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">return</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> new_state, metrics, new_dropout_rng</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">parallel_train_step = jax.pmap(train_step, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"batch"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">def</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #ffffaa; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">eval_step</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">(params, batch):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> labels = batch.pop(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"labels"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> logits = model(**batch, params=params, train=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">False</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">]</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> loss = optax.softmax_cross_entropy(logits[..., :</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">-1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">, :], onehot(labels[..., </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">:], logits.shape[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">-1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">])).mean()</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #888888; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"># summarize metrics</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> metrics = {</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"loss"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: loss, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"perplexity"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">: jnp.exp(loss)}</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> metrics = jax.lax.pmean(metrics, axis_name=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">"batch"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">return</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> metrics</span></p><p><span id="docs-internal-guid-5ee3f30e-7fff-4ad2-ae6b-7632dba1c01b"><br /><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">parallel_eval_step = jax.pmap(eval_step, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"batch"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><span id="docs-internal-guid-fe717cac-7fff-fc26-e05f-5534377c2492"></span><span id="docs-internal-guid-185ce34a-7fff-6e0d-fd41-9b1d7d7e91c4"><span style="font-family: Arial; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">17.複製參數到各個TPU上</span></span></p><p><span id="docs-internal-guid-84fd8ab3-7fff-a2d7-49bc-24da0e4f7c96"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">state = flax.jax_utils.replicate(state)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">rng = jax.random.PRNGKey(training_seed)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">dropout_rngs = jax.random.split(rng, jax.local_device_count())</span></span></p><p><span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">18.</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">開始訓練模型,把前面設定的內容都串接起來寫成迴圈</span></p><p><span id="docs-internal-guid-48421e1b-7fff-14ec-f474-cd725d9be39f"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> epoch </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> tqdm(range(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, num_epochs + </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">), desc=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">f"Epoch ..."</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, position=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, leave=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">):</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> rng, input_rng = jax.random.split(rng)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #888888; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"># -- Train --</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> train_loader = data_loader(input_rng, tokenized_datasets[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">], total_batch_size, shuffle=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">with</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> tqdm(total=len(tokenized_datasets[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"train"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]) // total_batch_size, desc=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"Training..."</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, leave=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">False</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">) </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">as</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> progress_bar_train:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> model_inputs </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> train_loader:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #888888; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"># Model forward</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> state, train_metric, dropout_rngs = parallel_train_step(state, model_inputs, dropout_rngs)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> progress_bar_train.update(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> progress_bar_train.write(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">f"Train... ({epoch}/{num_epochs} | Loss: {round(train_metric['loss'].mean(), </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">3</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)}, Learning Rate: {round(train_metric['learning_rate'].mean(), </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">6</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)})"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> )</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #888888; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"># -- Eval --</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> eval_loader = data_loader(input_rng, tokenized_datasets[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"validation"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">], total_batch_size)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> eval_metrics = []</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">with</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> tqdm(total=len(tokenized_datasets[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"validation"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">]) // total_batch_size, desc=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"Evaluation..."</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, leave=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">False</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">) </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">as</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> progress_bar_eval:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">for</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> model_inputs </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">in</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> eval_loader:</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #888888; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"># Model forward</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> eval_metric = parallel_eval_step(state.params, model_inputs)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> eval_metrics.append(eval_metric)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> progress_bar_eval.update(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">1</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> eval_metrics = get_metrics(eval_metrics)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> eval_metrics = jax.tree_map(jnp.mean, eval_metrics)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> progress_bar_eval.write(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">f"Eval... ({epoch}/{num_epochs} | Loss: {eval_metrics['loss']} | Perplexity: {eval_metrics['perplexity']})"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> )</span></span></p><p><span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">19.儲存模型</span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;"><br /></span></p><p><span id="docs-internal-guid-5837c53b-7fff-87ca-ebae-61c02ebf9e56"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">model.save_pretrained(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/model/'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span></span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">20.</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">(Optional)掛接雲端硬碟,讓程式可以存取雲端硬碟裡的內容,並將模型存到雲端硬碟的路徑裡</span></p><p><span id="docs-internal-guid-0b9a496e-7fff-d28b-4315-ab5e80bef76b"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> google.colab </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> drive</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">drive.mount(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/drive'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">your_path = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/your/path/'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">model.save_pretrained(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'/content/drive/MyDrive/{your_path}'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span></span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial;"><span style="font-size: 14.666667px; white-space: pre-wrap;">以上是訓練模型,下面兩個步驟則是推論,大家可以根據自己需求,把模型放到自家的GPU上做推論。</span></span></p><p><span style="font-family: Arial;"><span style="font-size: 14.666667px; white-space: pre-wrap;"><br /></span></span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">21.</span><span style="font-family: Arial; font-size: 11pt; white-space: pre-wrap;">推論-載入預訓練完的模型及tokenizer</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">由於程式執行保存檔案的路徑皆為colab的環境內,若是執行階段中斷則檔案也會一起被刪除。</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">如果有需要可以掛接到雲端硬碟,將檔案保存在雲端硬碟內,則不受執行階段影響,執行階段中斷之後也可以重新掛載雲端硬碟,並且使用檔案。</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial;"><span style="font-size: 14.666667px; white-space: pre-wrap;">訓練後我們可以直接在</span></span>transformers的幫助下進行推論</p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">from</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> transformers </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> AutoConfig, FlaxAutoModelForCausalLM, AutoTokenizer</span></p><p><span id="docs-internal-guid-ee851a19-7fff-0f49-b6fc-8c2dcc286d80"></span><span id="docs-internal-guid-cfde47e8-7fff-71b5-4e69-8d3b8a98928b"><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">import</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> jax.numpy </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">as</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> jnp</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">config = AutoConfig.from_pretrained(your_path)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">model = FlaxAutoModelForCausalLM.from_config(config, dtype=jnp.dtype(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"bfloat16"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">))</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">tokenizer = AutoTokenizer.from_pretrained(your_path)</span></span></p><p><br /></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">22.</span>推論-生成結果,藉由.generate function生成next token prediction的結果,再由tokenizer把預測還原為文字</p> <div><span id="docs-internal-guid-e4effaeb-7fff-b113-f2c2-93d8036c2c7c"><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">text = </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'要探討從梨俱吠陀到波你尼時代梵語的發展,可以考察'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">inputs = tokenizer(text, return_tensors=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"np"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">beam_output = model.generate(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> **inputs,</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> max_length=len(text)+</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">5</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">, </span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> early_stopping=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">print(</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">"Output:\n"</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> + </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">100</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"> * </span><span face="Consolas, sans-serif" style="background-color: #333333; color: #a2fca2; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">'-'</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">)</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;"><br /></span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">print(tokenizer.decode(beam_output.sequences[</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #d36363; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">0</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">], skip_special_tokens=</span><span face="Consolas, sans-serif" style="background-color: #333333; color: #fcc28c; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">True</span><span face="Consolas, sans-serif" style="background-color: #333333; color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">))</span></span><br /><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;"><br /></span></p><p><span style="font-family: Arial;"><span style="font-size: 14.666667px; white-space: pre-wrap;">而上面的text就是我們想問的問題,而下面的</span></span><span face="Consolas, sans-serif" style="background-color: #333333; caret-color: rgb(255, 255, 255); color: white; font-size: 14.666667px; white-space: pre-wrap;">tokenizer.decode</span><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">把我們答案給回傳出來。</span></p><p><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;">以上呢就是透過JAX來訓練原生GPT的過程囉。</span></p><p><span style="font-family: Arial;"><span style="font-size: 14.666667px; white-space: pre-wrap;">謝謝大家收看!</span></span></p><p><span style="font-family: Arial;"><span style="font-size: 14.666667px; white-space: pre-wrap;"><br /></span></span></p><p><span style="font-family: Arial;"><span style="font-size: 14.666667px; white-space: pre-wrap;"><br /></span></span></p><p><span style="font-family: Arial;"><span style="font-size: 14.666667px; white-space: pre-wrap;">當然如果您覺得自己弄太麻煩,也可以找Jerry老師團隊幫忙服務喔</span></span></p><p>GPT 代工一站式服務</p><p><a href="https://www.ap-mic.com/gpt">https://www.ap-mic.com/gpt</a></p><p>到這裡提交表單,即有專人會與您聯繫</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-Pf5jaBldsS9yMIoIZ6g1Xtw1BAxL5Q2CMC6xYFOQ9LZ-HtgsevKn-SwLp6ycsoa1a-aziM6LEpuVRl2rsUrb7AfNfXC-G7ipp3I5m-_GmF9bOUzfwt-E6tt_dGmvVvqg5Z8Ca-0pz5ceAs3fgctpRUz3mk17uOgccADksI1FX7Cg07Jwjqo6xePy/s1734/Screenshot%202023-05-19%20at%201.56.15%20PM.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1082" data-original-width="1734" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-Pf5jaBldsS9yMIoIZ6g1Xtw1BAxL5Q2CMC6xYFOQ9LZ-HtgsevKn-SwLp6ycsoa1a-aziM6LEpuVRl2rsUrb7AfNfXC-G7ipp3I5m-_GmF9bOUzfwt-E6tt_dGmvVvqg5Z8Ca-0pz5ceAs3fgctpRUz3mk17uOgccADksI1FX7Cg07Jwjqo6xePy/s320/Screenshot%202023-05-19%20at%201.56.15%20PM.png" width="320" /></a></div><br /><span style="font-family: Arial; font-size: 14.666667px; white-space: pre-wrap;"><br /></span><p></p></div>資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-54789234667081696402023-05-12T12:44:00.004+08:002023-05-12T12:51:24.201+08:00Google I/O 2023的速記<p><br /></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeSsUtG739oNmj0tc6K05rXbk74upD6GO_A3QxK6onk2A21t_peMyEXFsilMIOmcbDrJY06C_JIGpZ3ETEPB27YFV2ElVLH35bjOTazLAnZXSdiFuWI_xtpfafzfdV7EhwWsId42yWhflq_mhubrQfZy2b1ryEZsUPZ-gePp178UkIQTV7jcfbp_5b/s1724/Screenshot%202023-05-12%20at%2012.50.38%20PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1022" data-original-width="1724" height="380" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeSsUtG739oNmj0tc6K05rXbk74upD6GO_A3QxK6onk2A21t_peMyEXFsilMIOmcbDrJY06C_JIGpZ3ETEPB27YFV2ElVLH35bjOTazLAnZXSdiFuWI_xtpfafzfdV7EhwWsId42yWhflq_mhubrQfZy2b1ryEZsUPZ-gePp178UkIQTV7jcfbp_5b/w640-h380/Screenshot%202023-05-12%20at%2012.50.38%20PM.png" width="640" /></a></div><br /> 大家好我是Jerry老師,又來一年一度IO分享速記。<p></p><p>今年的Google I/O 2023,可以很清楚看到Google如同前幾年提到的方向,將AI部署到旗下所有產品當中,這一步棋,走的很早,但速度還不夠快。</p><p>但個人覺得因為面臨Microsoft加上GPT能力的產品提升,所以Google應該會增加整合速度,以及走的更深。從市場來看,過去台灣企業不知道如何導入AI,在未來台灣必定在各類系統走向AI化,已經是必然趨勢。</p><p>如果您想回顧去年的IO,請點這裡</p><p><a href="https://dataology.blogspot.com/2022/06/google-io-2022.html" target="_blank">Google IO 2022必須知道的十件事</a></p><p>這次IO可以看到Google將AI導入產品更加具體化,從各種功能上看起來都非常實用。</p><p>整體上IO重點發表了兩塊 AI、硬體,本篇針對AI的部分做比較多彙整。</p><p>Google從Gmail開始以生成式AI技術推出Smart Reply、Smart Compose、Help me write</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSIcCFTjRK1MnhPN0Bowkci9tFX1GeFGzdIVsieqtezl83ZmkVruASSbhtu13VX3k53BaD9_msyzL-suvaGpIGgBagPN64VHic1Z5S-fZLAoXlEGDbFoKQEhJHnoL4nEhrlGq5VJLI0wdHOsV9qbRa4LTU8pjrmanGn2TA16Tlj4ibg7gJRAfz9hf5/s1536/Screenshot%202023-05-12%20at%2012.30.28%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="846" data-original-width="1536" height="352" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSIcCFTjRK1MnhPN0Bowkci9tFX1GeFGzdIVsieqtezl83ZmkVruASSbhtu13VX3k53BaD9_msyzL-suvaGpIGgBagPN64VHic1Z5S-fZLAoXlEGDbFoKQEhJHnoL4nEhrlGq5VJLI0wdHOsV9qbRa4LTU8pjrmanGn2TA16Tlj4ibg7gJRAfz9hf5/w640-h352/Screenshot%202023-05-12%20at%2012.30.28%20PM.png" width="640" /></a></div><br /><p>Google Maps則持續更新Immersive View,去年提到也是採用機器學習建模讓街道成像能夠更真實。</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg79wzVCUaSJR_JYTEIW2VFr7u7Sem2QEvGHWlT4yaQUczxThXYJSn5J8oAlDPJ6ySbLYmueA0rYvLA1Z2FSKuAEltZPBUTsewRD1UlR3CwbyuD_ahLt4p3n86XJHUeKroso97A8aFhGGUpyOdcYji-LNhmHYEwr1zh4Tf0pT-avI_mdgCbff8rfV-e/s1570/Screenshot%202023-05-12%20at%2012.31.30%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="878" data-original-width="1570" height="358" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg79wzVCUaSJR_JYTEIW2VFr7u7Sem2QEvGHWlT4yaQUczxThXYJSn5J8oAlDPJ6ySbLYmueA0rYvLA1Z2FSKuAEltZPBUTsewRD1UlR3CwbyuD_ahLt4p3n86XJHUeKroso97A8aFhGGUpyOdcYji-LNhmHYEwr1zh4Tf0pT-avI_mdgCbff8rfV-e/w640-h358/Screenshot%202023-05-12%20at%2012.31.30%20PM.png" width="640" /></a></div><br /><p>再來是Google Photos提供了Find photos、Magic Eraser、Magic Editor</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7oo4v_xs6axE-x3NeM9twcrvRjG71LvnvIlb2hSyhci9ahHhKUue8cpR8IHctpwhNA-mraUnynHp2GeQkjtKLZrQy9klFw7vKsEqEZVyOE1a0X0knyTF-evbcD3PnVVKHi0smOUnP_yLEOBpn0JdYiAhfchxn3SWhZFKkqDdPtFA5LGpYbjsQcOiS/s1542/Screenshot%202023-05-12%20at%2012.32.09%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="962" data-original-width="1542" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7oo4v_xs6axE-x3NeM9twcrvRjG71LvnvIlb2hSyhci9ahHhKUue8cpR8IHctpwhNA-mraUnynHp2GeQkjtKLZrQy9klFw7vKsEqEZVyOE1a0X0knyTF-evbcD3PnVVKHi0smOUnP_yLEOBpn0JdYiAhfchxn3SWhZFKkqDdPtFA5LGpYbjsQcOiS/w640-h400/Screenshot%202023-05-12%20at%2012.32.09%20PM.png" width="640" /></a></div><br /><p>而上面的產品加值的技術源頭,就是PaLM2,事實上Jerry老師在4月的時候已經分享過<a href="https://gdg.community.dev/events/details/google-gdg-cloud-taipei-presents-palmneng-shuo-neng-kan-neng-dong-neng-li-jie-de-yu-yan-mo-xing/">PaLM是一個能說、能看、能動、能理解的語言模型</a></p><p>但PaLM缺點是目前支持語言並不多,之前Google所推出的語言模型技術當中,像是BERT、T5、MT5都普遍支援多國語言,以MT5為例,Jerry老師有介紹過 <a href="https://youtu.be/vvL3IyH6WeU" target="_blank">一個大規模跨國語言的預訓練模型MT5 </a>,就支援超過100種語言</p><p>至今MT5還是被用在做各種低成本的語言任務當中。Google最終還是要讓人工智能能夠幫助到每個人(Making AI helpful for everyone),讓人人都能受益。</p><p>在PaLM模型上,跟GPT一樣,他有推出不同的版本,包含Gecko、Otter、Bison、Unicorn</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqiFddyWvI9fRjkI35I-krUxIgG8LQRUPAkdEjdPEozEPxdf4-vSPK1p0TcOfA0_JcNaXCb2SBjmd3WAxkDYotA43h7S5yj0GJVa4oGA3GBZMszxUPxe4L5ER4SeF7qOZYLMDRrvdFVohAvTnyKYKvZWSTJ6o8aB0U4EVTLswTw5hXtbd93v9pMYPL/s1590/Screenshot%202023-05-12%20at%2012.33.18%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="758" data-original-width="1590" height="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqiFddyWvI9fRjkI35I-krUxIgG8LQRUPAkdEjdPEozEPxdf4-vSPK1p0TcOfA0_JcNaXCb2SBjmd3WAxkDYotA43h7S5yj0GJVa4oGA3GBZMszxUPxe4L5ER4SeF7qOZYLMDRrvdFVohAvTnyKYKvZWSTJ6o8aB0U4EVTLswTw5hXtbd93v9pMYPL/w640-h306/Screenshot%202023-05-12%20at%2012.33.18%20PM.png" width="640" /></a></div><br /><p>如果熟Google Cloud的朋友,就可以知道Bison可以到Vertex AI的Generative AI Studio功能中,已經上架供測試,但還是沒有多國語言啊啊啊....</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg30DiBmagMhyjcwsekKaBILrmt4CSEBHq945uHVnLdJF6OGG_JCTYTkTt3QmG35fQJtm_yUcQOhPY-Dne1jg8P9QP2mhxID5QcbyYuiVWoLnOZKH4t1IticLe7mT0TfmK6OJtGF3R6H2ckhq7BiBjYx5kVzlrDkAMbrpThVgPSgnYan7Ct1NxL6S79/s1596/Screenshot%202023-05-12%20at%2012.33.51%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1066" data-original-width="1596" height="428" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg30DiBmagMhyjcwsekKaBILrmt4CSEBHq945uHVnLdJF6OGG_JCTYTkTt3QmG35fQJtm_yUcQOhPY-Dne1jg8P9QP2mhxID5QcbyYuiVWoLnOZKH4t1IticLe7mT0TfmK6OJtGF3R6H2ckhq7BiBjYx5kVzlrDkAMbrpThVgPSgnYan7Ct1NxL6S79/w640-h428/Screenshot%202023-05-12%20at%2012.33.51%20PM.png" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsntBVXZL3hNJljvD1IAy8EQBvH_0NNyGOeZf5Kc4lFqjPS-UepRdQIQobwwJFbyjl2X-DiDzK3nY-DzmhQWH2Y7ST240rirD9OLJtljjoxKarJSn2KpIVN8CEir965SWp-3lh-Uhvh5D51npXrcmrjKs3idxYkv9liPWgA-QZqYOY8ltHzQ0qztRW/s982/Screenshot%202023-05-12%20at%2012.34.16%20PM.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="702" data-original-width="982" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsntBVXZL3hNJljvD1IAy8EQBvH_0NNyGOeZf5Kc4lFqjPS-UepRdQIQobwwJFbyjl2X-DiDzK3nY-DzmhQWH2Y7ST240rirD9OLJtljjoxKarJSn2KpIVN8CEir965SWp-3lh-Uhvh5D51npXrcmrjKs3idxYkv9liPWgA-QZqYOY8ltHzQ0qztRW/w400-h286/Screenshot%202023-05-12%20at%2012.34.16%20PM.png" width="400" /></a></div><br /><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p>也許還在測試?</p><p>但是Med-PaLM2會讓人覺得滿強大的,細節可以看<a href="https://www.youtube.com/watch?v=3Ud-BMOCkDI&list=PL590L5WQmH8e3dS9CtvRofb0nfdGb-Of9&index=4" target="_blank">這裡</a></p><p>他是全球第一個在美國醫學執照考試 (USMLE) 式問題的 MedQA 數據集上在“專家”應試者水平準確率達到 85% 以上的模型</p><p>接下來是大家期待的Bard,可以看到風格跟ChatGPT不太一樣,Bard會先把說明寫在前面,答案在後面。</p><p>但生成的程式碼結果其實不會差太多,預期上程式碼訓練來源應該都是差不多的地方。</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQEChF--uRJFdJrPui7mQcpAXycVU7BqK4jTT3-YQt7Sd2NUCQH5wh428Ncv9jF9c6MGa1YAwrehTHXMv0oGBk8DBM77sO-3TesqN_HMpgqjExJTXTVq_QDXqg1mByBZAQOybPMWkV5T69s5V1wlw5Cu4a8_TQjyhTdLqAkr_kBdG9iGmtFaiAnG7b/s1590/Screenshot%202023-05-12%20at%2012.35.24%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1304" data-original-width="1590" height="524" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQEChF--uRJFdJrPui7mQcpAXycVU7BqK4jTT3-YQt7Sd2NUCQH5wh428Ncv9jF9c6MGa1YAwrehTHXMv0oGBk8DBM77sO-3TesqN_HMpgqjExJTXTVq_QDXqg1mByBZAQOybPMWkV5T69s5V1wlw5Cu4a8_TQjyhTdLqAkr_kBdG9iGmtFaiAnG7b/w640-h524/Screenshot%202023-05-12%20at%2012.35.24%20PM.png" width="640" /></a></div><br /><p style="font-family: "PingFang TC"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">接下來我們可以看到<span style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal;">Google</span>的<span style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal;">Workspace</span>的部分,包含自動撰寫徵人資訊、自動試算表計算、試算表問答、簡報生成圖片素材等等,都是能夠增加生產力的功能,<span style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal;">Google</span>稱為<span style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal;">Duet AI for Workspace </span>一種協奏的概念:D</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtHj7tM88ByvUu9jIY0jvgOtZqCe3E_f1BoEovFdc95Xrgl41XeRwXircAs8qM-OkR0YD27QrMreMAXNIjEDUhEBz79yZJEvpCj5n3ldXlYZg6y-nSDD7DQaJTu-Zy1dyuNBBsGoJcBOmtWLAgRwdHVDna8GzIVxqrnxm3LKBa1QFSAu7MTN7Y6m9W/s1588/Screenshot%202023-05-12%20at%2012.36.02%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="956" data-original-width="1588" height="386" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtHj7tM88ByvUu9jIY0jvgOtZqCe3E_f1BoEovFdc95Xrgl41XeRwXircAs8qM-OkR0YD27QrMreMAXNIjEDUhEBz79yZJEvpCj5n3ldXlYZg6y-nSDD7DQaJTu-Zy1dyuNBBsGoJcBOmtWLAgRwdHVDna8GzIVxqrnxm3LKBa1QFSAu7MTN7Y6m9W/w640-h386/Screenshot%202023-05-12%20at%2012.36.02%20PM.png" width="640" /></a></div><br /><p>再來是Google最強的搜尋產品,看起來新功能是直接在綠色文字區塊呈現,而非Bing的方式是放在右邊,從UI/UX來看Google的作法更為細緻與深入些,因為直接推薦問題,問答的互動則是再細部資料呈現之後再提供</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_St2BHxYn5S3_lXCqgTWVSYVIvog5wWmxyMZ1JOrKG1dri5xBZHWmRMpzRL0CdZ2AVn1fzzniPDm0aaQPhaw6cTvnH4535AcI-u9QY40AJamLinVoAN_lnPfBRokpRxpTb0VS5bNP4aNnr_LTFP3ks10nzoeMzoFDyZDHLavSPA4N2tplv0vLj136/s1584/Screenshot%202023-05-12%20at%2012.36.40%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="968" data-original-width="1584" height="392" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_St2BHxYn5S3_lXCqgTWVSYVIvog5wWmxyMZ1JOrKG1dri5xBZHWmRMpzRL0CdZ2AVn1fzzniPDm0aaQPhaw6cTvnH4535AcI-u9QY40AJamLinVoAN_lnPfBRokpRxpTb0VS5bNP4aNnr_LTFP3ks10nzoeMzoFDyZDHLavSPA4N2tplv0vLj136/w640-h392/Screenshot%202023-05-12%20at%2012.36.40%20PM.png" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjV88aiork8DwRl-NsDfPXNOQBbK5O83r-j0zZV-4ukiebifpyGWIONY-siYHpLlad3RTLVbVtxjD3TYtJxraxPbwxY16acUyCTX57cytf3iBZwo0-zNSfpvKbg-uDXPGLWcmz6HP7Uxv7-Wlh1Y3QPJIdoNJvYLDD80k_PXkUla3ikW97Pidcg3bt_/s1584/Screenshot%202023-05-12%20at%2012.37.15%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1026" data-original-width="1584" height="414" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjV88aiork8DwRl-NsDfPXNOQBbK5O83r-j0zZV-4ukiebifpyGWIONY-siYHpLlad3RTLVbVtxjD3TYtJxraxPbwxY16acUyCTX57cytf3iBZwo0-zNSfpvKbg-uDXPGLWcmz6HP7Uxv7-Wlh1Y3QPJIdoNJvYLDD80k_PXkUla3ikW97Pidcg3bt_/w640-h414/Screenshot%202023-05-12%20at%2012.37.15%20PM.png" width="640" /></a></div><br /><p>看了覺得很讚,但還是一樣,希望中文能夠盡快推進。</p><p><br /></p><p>再來是Google Cloud介紹,新的PaLM API的案例、NVIDIA H100的機器推出、Colab推出程式自動撰寫功能等。</p><p>還有比較特別的是語言轉換影片</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLYtneKYCHa53dmsvhGAWKNIzKJL4-82nhGDm7IrBHU4PxgG1YYYFFqvjI1MyMH3kWZQE2VA1uFel_akSErCW7_a9WQMfUZTyBdU6k0I8s6PODOJY-aC4ys549Q3ymrLzmPetD2Ami4WCQ3eDdYppKhziAbtJXR8Su8EkMoL7R1_gcnX-LzSibG_Wi/s1592/Screenshot%202023-05-12%20at%2012.37.53%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="746" data-original-width="1592" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLYtneKYCHa53dmsvhGAWKNIzKJL4-82nhGDm7IrBHU4PxgG1YYYFFqvjI1MyMH3kWZQE2VA1uFel_akSErCW7_a9WQMfUZTyBdU6k0I8s6PODOJY-aC4ys549Q3ymrLzmPetD2Ami4WCQ3eDdYppKhziAbtJXR8Su8EkMoL7R1_gcnX-LzSibG_Wi/w640-h300/Screenshot%202023-05-12%20at%2012.37.53%20PM.png" width="640" /></a></div><div>可以看到從英文轉為另一個語言,甚至連嘴型都可以配合。透過Lip matching 、 Speech generation、Intonation、Translation技術來達到這個效果。</div><div><br /></div><div>後面的部分則是介紹Android生態,以及未來硬體+AI的成長變化</div><div>像是在Android手機、平板上可以方便使用搜尋、工作甚至是Bard的整體介面。</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqkqCY8F3pLjwgw2mRdEjcInYXhhYU5lgZqxNFsYFSa227KCfVG7Q8r5v05-Y393JjBYlZfGkrquzp0cFJJTU_rl48PzYks16C2dkckQInRHar-Q0Z9QCb83-TIIyiqv-IeiHP55zjZakaa7oPuNQenFtzbT0RZtG9HX370fgGZgZjjA1Gjtm9izdn/s1600/Screenshot%202023-05-12%20at%2012.38.28%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="902" data-original-width="1600" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqkqCY8F3pLjwgw2mRdEjcInYXhhYU5lgZqxNFsYFSa227KCfVG7Q8r5v05-Y393JjBYlZfGkrquzp0cFJJTU_rl48PzYks16C2dkckQInRHar-Q0Z9QCb83-TIIyiqv-IeiHP55zjZakaa7oPuNQenFtzbT0RZtG9HX370fgGZgZjjA1Gjtm9izdn/w640-h360/Screenshot%202023-05-12%20at%2012.38.28%20PM.png" width="640" /></a></div><br /><p>接下來針對幾個Google推出的AI服務,Jerry老師也做一下彙整</p><p>對於Generative AI有興趣的</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9xe44sK8rqY1n3KJMqJ5zDQbvRcqACH3Fb4kLlAGv_M50iLX9YrAcRM5aYEKO8Ki58HDUsj4mqgffaS-ndweIkX5wX3J_AilCqdRoT11ZK0sujyc2MnIPM2O4mK8OQpTOf9pS3Py-156_MKo0XS_ulLqNAScllUtxOmp3Id937I2Ace68mpxGJLBh/s1598/Screenshot%202023-05-12%20at%2012.39.05%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="826" data-original-width="1598" height="330" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9xe44sK8rqY1n3KJMqJ5zDQbvRcqACH3Fb4kLlAGv_M50iLX9YrAcRM5aYEKO8Ki58HDUsj4mqgffaS-ndweIkX5wX3J_AilCqdRoT11ZK0sujyc2MnIPM2O4mK8OQpTOf9pS3Py-156_MKo0XS_ulLqNAScllUtxOmp3Id937I2Ace68mpxGJLBh/w640-h330/Screenshot%202023-05-12%20at%2012.39.05%20PM.png" width="640" /></a></div><br /><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">可以參閱網址:</p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/21666103-ad8f-463f-9f05-a2b1e11aeb60/">https://io.google/2023/program/21666103-ad8f-463f-9f05-a2b1e11aeb60/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/ca897c89-e56a-445c-97d6-eda2df5adecc/">https://io.google/2023/program/ca897c89-e56a-445c-97d6-eda2df5adecc/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/27cce05f-df4c-4ab2-9a59-5b466bdae0f9/">https://io.google/2023/program/27cce05f-df4c-4ab2-9a59-5b466bdae0f9/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/79bf6ddf-781f-4b99-a7f7-6202df00052d/">https://io.google/2023/program/79bf6ddf-781f-4b99-a7f7-6202df00052d/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">今年的LLMs 與 生成式AI的主題相當多,只好有空再來整理給大家了</p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="font-family: "PingFang TC"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">跟生成式AI有關的輸出責任問題<span style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal;"> </span>可以參考這個</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghsLhLDArlRJoOAur7XZHwRQjUYZT_Cd4aNVaYORGt5FB0C2QWBLhiX8xjTFohg0rhd87iROoDDcU-eDQhC2ugN_yvsBykWeSYA1j9AeRrubwX1ucZgYXxKA1mNIyeHc6yP1Xr6dMEPA0_ljeIS1oCOwndccsM0xIwKmjDrmfwKrYhKEu0unsVJOfn/s1598/Screenshot%202023-05-12%20at%2012.39.35%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="646" data-original-width="1598" height="258" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghsLhLDArlRJoOAur7XZHwRQjUYZT_Cd4aNVaYORGt5FB0C2QWBLhiX8xjTFohg0rhd87iROoDDcU-eDQhC2ugN_yvsBykWeSYA1j9AeRrubwX1ucZgYXxKA1mNIyeHc6yP1Xr6dMEPA0_ljeIS1oCOwndccsM0xIwKmjDrmfwKrYhKEu0unsVJOfn/w640-h258/Screenshot%202023-05-12%20at%2012.39.35%20PM.png" width="640" /></a></div><br /><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">概念上就是多了一層輸出的模型處理答案的正確與否,這跟以前的LaMDA有做一個分類跟評分的「流程」很像。</p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/04c03aa2-e9ac-44f8-865a-33c1a494e11d/">https://io.google/2023/program/04c03aa2-e9ac-44f8-865a-33c1a494e11d/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/5668793d-5e8b-4da1-8813-5d2972e0270a/">https://io.google/2023/program/5668793d-5e8b-4da1-8813-5d2972e0270a/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">對AR有興趣可以看這裡</p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/0192be73-9c6a-46b2-bd1e-d9a4818c1cc8/">https://io.google/2023/program/0192be73-9c6a-46b2-bd1e-d9a4818c1cc8/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/750f0a8b-bab2-41f3-9cef-61d02f2369f3/">https://io.google/2023/program/750f0a8b-bab2-41f3-9cef-61d02f2369f3/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://io.google/2023/program/e0949a49-5d57-4551-9b41-1650536a5800/">https://io.google/2023/program/e0949a49-5d57-4551-9b41-1650536a5800/</a></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">另外直得深入研究的是</p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">Kaggle Model 這個跟 huggingface有得比的模型庫,而且是歷經各種挑戰的模型庫</p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://www.kaggle.com/models">https://www.kaggle.com/models</a></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwjusjpcw5ySIvR2ny9R08SgkefAo8J8KwZ8cW0H5OKwChUq7N1PBxWAzxKSRBGM4QOIPNby9ONlTsULJqRPCaf8xW_3RLazPqjwtd62PonOyqA43qMHZzsVvL1xVMfjmr2YLGaLfavcU86goFfjeAPNgRsTLcvrW7wCK0OzFaNTLGEe19cryDOFJ0/s1586/Screenshot%202023-05-12%20at%2012.40.37%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="902" data-original-width="1586" height="364" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwjusjpcw5ySIvR2ny9R08SgkefAo8J8KwZ8cW0H5OKwChUq7N1PBxWAzxKSRBGM4QOIPNby9ONlTsULJqRPCaf8xW_3RLazPqjwtd62PonOyqA43qMHZzsVvL1xVMfjmr2YLGaLfavcU86goFfjeAPNgRsTLcvrW7wCK0OzFaNTLGEe19cryDOFJ0/w640-h364/Screenshot%202023-05-12%20at%2012.40.37%20PM.png" width="640" /></a></div><br /><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">以及在Google Cloud上面的模型</p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><a href="https://console.cloud.google.com/vertex-ai/model-garden">https://console.cloud.google.com/vertex-ai/model-garden</a></p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzovyh1kMV2QfwJ1GNTEXxSUPjtpQyqZUrjPBoGYm18JoIEk6k37Vx_ARYRwsTGwyqNP8dCfLMoBpcDc4orlP5huUaSooZGyZNPRGoGZnkwg8_dsYgsD9SWvRGDa5qqxrnCcMCAVGHOBDp5x7v2RzhhJu7uiaQhGbGPu6mCx-LPYsG9WjByJ8rvJVa/s1596/Screenshot%202023-05-12%20at%2012.41.04%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="792" data-original-width="1596" height="318" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzovyh1kMV2QfwJ1GNTEXxSUPjtpQyqZUrjPBoGYm18JoIEk6k37Vx_ARYRwsTGwyqNP8dCfLMoBpcDc4orlP5huUaSooZGyZNPRGoGZnkwg8_dsYgsD9SWvRGDa5qqxrnCcMCAVGHOBDp5x7v2RzhhJu7uiaQhGbGPu6mCx-LPYsG9WjByJ8rvJVa/w640-h318/Screenshot%202023-05-12%20at%2012.41.04%20PM.png" width="640" /></a></div><br /><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">相信在未來做機器學習的學習成本會越來越低,更快導入到各行各業。</p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;">如果您有看到什麼不錯,想跟Jerry老師分享的</p><p style="color: #dca10d; font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span style="color: black;">也歡迎來信 <a href="mailto:dsjerry2017@gmail.com">dsjerry2017@gmail.com</a></span></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px; min-height: 15px;"><br /></p><p style="font-family: "Helvetica Neue"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-size: 13px; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal; margin: 0px;"><span style="font-family: "PingFang TC"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal;">以上就是這次</span>Google IO 2023<span style="font-family: "PingFang TC"; font-feature-settings: normal; font-kerning: auto; font-optical-sizing: auto; font-size-adjust: none; font-stretch: normal; font-style: normal; font-variant-alternates: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-position: normal; font-variation-settings: normal; line-height: normal;">的報告 謝謝大家</span></p>資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-2170097852409651912022-11-02T17:06:00.012+08:002022-11-03T14:47:07.033+08:00Google Machine Learning Community Summit 2022 活動速記<p><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHMpAwwa-YVrWVm_eB5GX0-yQQloOCw9zM26a7AQimAuosRERCx0rf9elEta0As8Gum1Mo-ef-1Z4ay3kI1nFngROyoJr3y_hEnCokJCyatmHvgOedcN7ewwpdE1iVXE4XUvqDNFkBW0ngrIDosnk0mKZD1NhE0EwP6YfkbwA6ZCbyB__h5EImuEm_/s2734/1.jpeg" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="1538" data-original-width="2734" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHMpAwwa-YVrWVm_eB5GX0-yQQloOCw9zM26a7AQimAuosRERCx0rf9elEta0As8Gum1Mo-ef-1Z4ay3kI1nFngROyoJr3y_hEnCokJCyatmHvgOedcN7ewwpdE1iVXE4XUvqDNFkBW0ngrIDosnk0mKZD1NhE0EwP6YfkbwA6ZCbyB__h5EImuEm_/w400-h225/1.jpeg" width="400" /></a><span style="text-align: center;">照片來自Sakon</span></p><p><br /></p><p>這次在疫情之後,第一次實體到訪泰國參與 Google 所舉辦的ML Community Summit 2022 這個活動一連舉辦兩天,從中間可以學習最新的產品的訊息,包含最新產品資訊包含Vertex AI、JAX, TPU, 圓桌討論、Team building活動等</p><p>加入Google Developer Expert Program之後,最多的收穫就是跟來自不同國家的交流,這次出席的開發專家有來自印度、韓國、土耳其、美國、英國、新加坡、臺灣、摩洛哥、巴西、義大利、哥倫比亞、法國、馬來西亞(按照簡報順序排序)等國家。</p><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><p><br /></p><p>第一天見面禮超讚</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii6P2u-JcdxZhhqDhrO2H6T-m1ktGzhhSIkR3bE__uHzEngQ0EhduEnOzgRatKvsIWrdZHY7nSKiPl2SnTrKZtaADo4BfKUQNozZzfuWC7nYwF4kMQnrPjiLCMNom9c8LJHDEDKP2DRFdqa4Rgna09bl8XG_h15SYFnGEQikI2s8O5hjwC18v7OBua/s1899/20221022_0814192.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="1899" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii6P2u-JcdxZhhqDhrO2H6T-m1ktGzhhSIkR3bE__uHzEngQ0EhduEnOzgRatKvsIWrdZHY7nSKiPl2SnTrKZtaADo4BfKUQNozZzfuWC7nYwF4kMQnrPjiLCMNom9c8LJHDEDKP2DRFdqa4Rgna09bl8XG_h15SYFnGEQikI2s8O5hjwC18v7OBua/w400-h300/20221022_0814192.jpeg" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Sakon</div><p>開場的時候我們的ML計畫主持人,幫大家準備了驚喜,就是<a href="https://en.wikipedia.org/wiki/Jeff_Dean">Jeff Dean</a> 幫大家親自簽名的論文,真的很榮幸能夠拿到他親自發表的論文以及簽名</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLhJo8tRIi7bCJM7AlXERQSrSSXsyc6gupCbTxb_7NraFe4ZvjA5pb-Ykoiky4T2Ym64v0RByw3A0KuGjxLklQU0xJjRqajtljhbezBB0v-Y1S5WauQ-BgXez1e1vEAUL6AvoqQEpTf7a9MlwZj20FjeyaH34quVCZENbLkHe9RfGeX80CyX7A-UJc/s1424/PXL_20221022_0224565292.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="1072" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLhJo8tRIi7bCJM7AlXERQSrSSXsyc6gupCbTxb_7NraFe4ZvjA5pb-Ykoiky4T2Ym64v0RByw3A0KuGjxLklQU0xJjRqajtljhbezBB0v-Y1S5WauQ-BgXez1e1vEAUL6AvoqQEpTf7a9MlwZj20FjeyaH34quVCZENbLkHe9RfGeX80CyX7A-UJc/w301-h400/PXL_20221022_0224565292.jpeg" width="301" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Sakon</div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQL0-0YoO9QnUARzjSnnW9mqHaKqXLJovn7s8Zf18x_6byBgLy1RElvCkOjmSDNmyyTqQ8KZ3UX_UDS6gSbinY1PWxxQYabQEZdv2RCGkosiukSoZK09j9WCC7rOVLCZxFNLP4m5CJPbC-dkvp7gz_iZbXSOtSS6tDaKNugL5Rq6PtGiP-KLkUUmVg/s1424/4.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="801" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQL0-0YoO9QnUARzjSnnW9mqHaKqXLJovn7s8Zf18x_6byBgLy1RElvCkOjmSDNmyyTqQ8KZ3UX_UDS6gSbinY1PWxxQYabQEZdv2RCGkosiukSoZK09j9WCC7rOVLCZxFNLP4m5CJPbC-dkvp7gz_iZbXSOtSS6tDaKNugL5Rq6PtGiP-KLkUUmVg/w225-h400/4.jpeg" width="225" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Sakon</div><p>真的很棒。</p><p><br /></p><p>接著就是陸續專家分享機器學習當紅議題,當然目前Google Cloud上面最流行的機器學習工具是Vertex AI,Google邀請了AI Platform Core的專家,從MLOps的角度做了完整的介紹,以及產品的發展方向,當然這個有簽NDA就不能透露太多,總之Vertex AI會是一個雲端在機器學習上會很便利的工具,當然這要結合GCP才能發揮對應價值,</p><p>有興趣可參閱 這個網址與影片</p><p>Vertex AI </p><p><a href="https://cloud.google.com/vertex-ai">https://cloud.google.com/vertex-ai</a></p><p>What is Vertex AI? </p><p><a href="https://youtu.be/gT4qqHMiEpA">https://youtu.be/gT4qqHMiEpA</a></p><p>如果您想要自己弄一個,當然也有開源的,可以參考Google開源的Kubeflow, TFX</p><p><a href="https://www.kubeflow.org/">https://www.kubeflow.org/</a></p><p>以及</p><p><a href="https://www.tensorflow.org/tfx">https://www.tensorflow.org/tfx</a></p><p>kubeflow + TFX + JupyterLab再搭配基礎設施,就能弄一個類似的東西,就端看我們要快速便利,還是要自己弄,Jerry老師認為能否自己弄的關鍵,應該就是團隊強不強,再來就是管理工具要自己花時間設計與開發囉。</p><p><br /></p><p>接著是現在<a href="https://www.deepmind.com/blog/using-jax-to-accelerate-our-research">Google DeepMind</a>團隊用很兇的<a href="https://github.com/google/jax">JAX</a>,您可以想像他是Python Numpy當中的進階版,因為它可以直接支援GPU跟TPU,Jerry老師在讀書會中有一個簡單分享,請參閱這裡</p><p><a href="https://youtu.be/SllsasgVntg">https://youtu.be/SllsasgVntg</a></p><p>以現行的各種AI演算法,最吃重的運算資源,需要快速的透過程式碼去操控,但是目前很多框架都是包起來,所以都要透過工具去觀察狀況,但是不一定能夠做很快速的修改,Jerry老師認為JAX的強項在於,把很多要做深度學習的事情函式化了,像是vmap可以輕鬆實現向量化的操作,把很多維度的工作處理得很簡單,還有很多其他像是jit、grad等功能,網路上有人寫了鐵人賽</p><p><a href="https://ithelp.ithome.com.tw/users/20129616/ironman/5220">https://ithelp.ithome.com.tw/users/20129616/ironman/5220</a></p><p>有興趣可以參考一下</p><p>最後就是TPU,這個高技術含量的議題,大家都知道AI在算力上一直都是一個重要的議題,因為所有模型除了「訓練」之外,連「推論」基本上都已經要GPU進行,在這次活動,Jerry老師看到了Google對TPU研發的實力,包含即將推出的Cloud TPU v4,可以超過250 teraflops(萬億次浮點運算每秒),重點是耗電量比原來2018年推出的Cloud TPU v3還少了30%~40%真的很不容易,處理頻寬可以到PB等級,實在是很讓人期待。</p><p>有興趣的朋友可以看這部影片</p><p>Cloud TPU v4: Fast, flexible, and easy-to-use ML accelerators</p><p><a href="https://youtu.be/W7A-9MYvPwI">https://youtu.be/W7A-9MYvPwI</a></p><p>中午Jerry老師就跟一群來自印度的朋友一起吃飯,分享自己的心得。</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbk0uYl7wGJM2Z17XWqA6BqyltpnLzdqk9PbOIXiHXSOxjRnrWaMEQdAGQN704IKIUuFFBDmgjMdJSw7h91WPl_cWphO90WF_FdcrANisT813uN2ZGasEHHwSy_TArmVc9v22SbmuAQ5c_wFLLAfEJU204RS6CgE7jlpdE7psMbZMOwtE0TDQBIKrK/s2136/5.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="2136" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbk0uYl7wGJM2Z17XWqA6BqyltpnLzdqk9PbOIXiHXSOxjRnrWaMEQdAGQN704IKIUuFFBDmgjMdJSw7h91WPl_cWphO90WF_FdcrANisT813uN2ZGasEHHwSy_TArmVc9v22SbmuAQ5c_wFLLAfEJU204RS6CgE7jlpdE7psMbZMOwtE0TDQBIKrK/w400-h266/5.jpeg" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Sakon</div><br /><p>下午就是輪流報告GDE在各國的狀況以及自己的動態等</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGF3e4ycgypXvboWSK3L32chJ4tLRUbTH0NYT0qtjDAqZaq7S1mYSYJGhQlB5KeBvFEuoB_YBVrOcNUaSbW5JK9WaTaewTDdEzCwndbK2AP86xTA-2l5jtji6i0WFALqBJRPsMfyBfId2L4lnTlZG95gwgOR73QdSaxYIwliUyG5BpzD7QZGretm10/s2136/6.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="2136" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGF3e4ycgypXvboWSK3L32chJ4tLRUbTH0NYT0qtjDAqZaq7S1mYSYJGhQlB5KeBvFEuoB_YBVrOcNUaSbW5JK9WaTaewTDdEzCwndbK2AP86xTA-2l5jtji6i0WFALqBJRPsMfyBfId2L4lnTlZG95gwgOR73QdSaxYIwliUyG5BpzD7QZGretm10/w400-h266/6.jpeg" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Sakon</div><br /><p>再來就是圓桌會議,討論未來大家的方向</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieQOadPXcX5peZ3UrS5h4Rf7EEq07k9jkUaQ4bHylw2yU42sbmhZrVIUZcxqO6wuubmer7NhVWnnjX-igmRvKSg6W8SBf_YY8vKkQFYJHUMcYPezV22E1_Dv6QS0kevHt2gE3uiKsV4dyD6JyEihGJjFTv9OaYQnOzwwE-JKApp606xXKC93AFbp25/s500/8.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="375" data-original-width="500" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieQOadPXcX5peZ3UrS5h4Rf7EEq07k9jkUaQ4bHylw2yU42sbmhZrVIUZcxqO6wuubmer7NhVWnnjX-igmRvKSg6W8SBf_YY8vKkQFYJHUMcYPezV22E1_Dv6QS0kevHt2gE3uiKsV4dyD6JyEihGJjFTv9OaYQnOzwwE-JKApp606xXKC93AFbp25/w640-h480/8.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自<a href="https://medium.com/google-developer-experts/ml-community-summit-2022-4567dc63de71">Margaret</a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZIqYSRg_TVM_X9YONLKZa_zN6HmdGL7Fvn36V2Z318zVvegQMhfjht0wWsOVDbzJ_hw8JIYcgogGcBQe-XzyCQyunyR7nrAYHRsgX8OQ33gGqnhOQ3gpqzeEc18i0eLwo6Ck1rlEIYfmNJx2H326Eei1CbXmDrG0EJbczZEgDjmlxOot-cc_nA6Z5/s2136/7.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="2136" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZIqYSRg_TVM_X9YONLKZa_zN6HmdGL7Fvn36V2Z318zVvegQMhfjht0wWsOVDbzJ_hw8JIYcgogGcBQe-XzyCQyunyR7nrAYHRsgX8OQ33gGqnhOQ3gpqzeEc18i0eLwo6Ck1rlEIYfmNJx2H326Eei1CbXmDrG0EJbczZEgDjmlxOot-cc_nA6Z5/w640-h426/7.jpeg" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Sakon</div><p>第二天我們參觀了泰國幾個重要的廟宇以及運河,很開心跟來自Hugging Face的專家Merve還有Margaret以及幾個同車的各國朋友一起互動,最開心的是拿到了遠從法國搭飛機過來的貼紙,Merve她是<a href="https://huggingface.co/">Hugging Face</a>的法國NLP專家</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBTC0tBNCsQ6-y4R0jVBcbBLRhw3Ed9EXIM1YemZggCs__d1lw4LdWxkZlxjC0hPN6XOdCe5RFLBFuLn7XLQn2g4Ay6d_zfX3GipUUOBrvglngd4hbVq7pUqa5Grjlm4FCcJTb24zVqGUAVad-Zlx2EgxagI-WV-DQTTySr2Mp9OtHtccVyijlNkyF/s1424/999.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="1068" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBTC0tBNCsQ6-y4R0jVBcbBLRhw3Ed9EXIM1YemZggCs__d1lw4LdWxkZlxjC0hPN6XOdCe5RFLBFuLn7XLQn2g4Ay6d_zfX3GipUUOBrvglngd4hbVq7pUqa5Grjlm4FCcJTb24zVqGUAVad-Zlx2EgxagI-WV-DQTTySr2Mp9OtHtccVyijlNkyF/w300-h400/999.jpeg" width="300" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Jerry</div><p><br /></p><p>大家可以到這裡看到她的風采</p><p><a href="https://huggingface.co/merve">https://huggingface.co/merve</a></p><p><a href="https://www.youtube.com/watch?v=IBaDGxgY3Po">https://www.youtube.com/watch?v=IBaDGxgY3Po</a></p><p><a href="https://github.com/merveenoyan">https://github.com/merveenoyan</a></p><p>以及跟一樣自然語言專業的新加坡同行交流了虛擬人的市場與技術</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYGSw1i638PamyWQLX5qNw7DgK0jLQ45Z7aJag-ltQoOumz2fq6aSwFd_fioorN5Z5S0TXVHeKSLls9-6kw3Mc2RxBkQuwIdp7Syb_FSJ70z9VrnKXqYr-ZGQHw82hmABeptjx0JgBzwo5I-0X4FmVGvs3bGBa49PiuG71Uo8lF_d9aNdiy_UV9jS1/s2136/10.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="2136" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYGSw1i638PamyWQLX5qNw7DgK0jLQ45Z7aJag-ltQoOumz2fq6aSwFd_fioorN5Z5S0TXVHeKSLls9-6kw3Mc2RxBkQuwIdp7Syb_FSJ70z9VrnKXqYr-ZGQHw82hmABeptjx0JgBzwo5I-0X4FmVGvs3bGBa49PiuG71Uo8lF_d9aNdiy_UV9jS1/w640-h426/10.jpeg" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Sakon</div><p><br /></p><p>最後我們有一個完美的合照</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiym2R9DVR3FgnhTFDRBGtB84UP32Aj0cPX9ciuwSKKNn0rsbvYbT7lhxoA-Z2GpzM84_oZ3V0g5JWpZ3ANJdT6Oy9GEWFAxs9Pyu1kvtHDfEN4GdTtCrlQlXs-V41Kuo_GWX9dzP1uuEWXutxx-CZdvUCciqH9olDkv1Xou9BhZ2yQ7RXxPrJ5SKj/s1899/11.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="1899" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiym2R9DVR3FgnhTFDRBGtB84UP32Aj0cPX9ciuwSKKNn0rsbvYbT7lhxoA-Z2GpzM84_oZ3V0g5JWpZ3ANJdT6Oy9GEWFAxs9Pyu1kvtHDfEN4GdTtCrlQlXs-V41Kuo_GWX9dzP1uuEWXutxx-CZdvUCciqH9olDkv1Xou9BhZ2yQ7RXxPrJ5SKj/w640-h480/11.jpeg" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Sakon</div><br /><p>跟各國專家互動交流,除了可以學習到不同的國家文化、談吐、價值觀,更可以了解全球大家對於機器學習這個技術的發展看法,以及最新的做法,最後恭喜下面中間這位印度朋友Sayak正式加入Hugging Face成為Developer Advocate Engineer。</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQmBrknKbImoz0RSVNCCeRcaqL7PuW4At5AKp01eTFA7D3XEENFNJCjo2aF-DlFulTWAD9KdTd_jckqCfkobgfZE0tRJf_HIZmF8TyB8iOWV0TZsaH7L5Ppsbirnt6W14B20jF8P9DWI13_fA6FDAHnqPv1q3N4QNn067CriEqSKCLERPBQn4xBFyv/s1424/122.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1424" data-original-width="674" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQmBrknKbImoz0RSVNCCeRcaqL7PuW4At5AKp01eTFA7D3XEENFNJCjo2aF-DlFulTWAD9KdTd_jckqCfkobgfZE0tRJf_HIZmF8TyB8iOWV0TZsaH7L5Ppsbirnt6W14B20jF8P9DWI13_fA6FDAHnqPv1q3N4QNn067CriEqSKCLERPBQn4xBFyv/w189-h400/122.jpeg" width="189" /></a></div><div class="separator" style="clear: both; text-align: center;">照片來自Jerry</div><br /><p>您也可以在這裡看到Sayak,他也是一個知名的ML專家</p><p><a href="https://sayak.dev/">https://sayak.dev/</a></p><p><a href="https://youtu.be/Q6BqBlUN8Lc">https://youtu.be/Q6BqBlUN8Lc</a></p><p>Hugging Face儼然已經成為機器學習的中立國家,大家快點加入台灣的Hugging Face群組</p><p><a href="https://huggingface.co/TFUGT">https://huggingface.co/TFUGT</a></p><p>為台灣爭光:D</p><p>最後也很榮幸認識Margaret,她也算是相當活躍的ML專家,同時具備藝術專長,可以到這裡看到她資訊 <a href="https://margaretmz.art/">https://margaretmz.art/</a></p><p>還有來自英國的NLP專家<a href="https://www.linkedin.com/in/grigorysapunov/">Grigory</a> 剛好要在2023年出全球第一本JAX的書,有興趣也可以參考一下</p><p><a href="https://www.manning.com/books/jax-in-action">https://www.manning.com/books/jax-in-action</a></p><p>對JAX想研究可以參考他的medium</p><p><a href="https://moocaholic.medium.com/jax-a13e83f49897">https://moocaholic.medium.com/jax-a13e83f49897</a></p><p><br /></p><p>請大家加入台灣Google開發者社群行列</p><p><a href="https://developers.google.com/">https://developers.google.com/</a></p><p>還有關注全球的開發者活動</p><p><a href="https://developers.google.com/community/devfest">https://developers.google.com/community/devfest</a></p><p>如果對Google專家計畫有興趣可以參考<a href="https://ericsk.medium.com/%E7%82%BA%E4%BB%80%E9%BA%BC%E4%BD%A0%E6%87%89%E8%A9%B2%E7%94%B3%E8%AB%8B%E6%88%90%E7%82%BA-google-developers-experts-gde-58cf7c361f62">這裡</a></p><p>還有太多資訊可以說了,但是工作太多,只能說收穫滿滿,只好改天找機會再跟大家分享囉。</p><p>下次見!</p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p>資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0台灣23.69781 120.960515-7.9761215634848384 85.804265 55.371741563484839 156.116765tag:blogger.com,1999:blog-5898372732399324805.post-29502148846246261332022-06-09T18:37:00.006+08:002022-06-29T12:19:08.512+08:00Google IO 2022必須知道的十件事<p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">今年Google IO 2022在<span lang="EN-US">AI 上</span>必須知道的十件事<o:p></o:p></p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><o:p> </o:p></p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">大家好我是Jerry老師,已經有一段時間因為公司忙碌沒發文,今天要跟大家分享的是關於今年Google在Google I/O的活動上分享最近透過AI方法做的十件事,特別在Google產品上都透過AI做了許多附加價值的事情,包含在Google翻譯、Google地圖、Youtube、Google Meet、Google Doc、Google搜尋、 Google助理、 Google手機、Google眼鏡等9<span lang="EN-US">種產品上,全文參考內容來自於<a href="https://io.google/2022/program/8e80903f-955f-4a5b-9118-b0ce4acdb0e6/" target="_blank">這裡</a></span><o:p></o:p></p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><o:p> </o:p></p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">今年的開頭讓Jerry老師覺得很棒,因為疫情、環境的關係,Google做了很多對社會有很大幫助的事情,像是「預測洪水的預警時間,發送超過2300萬則的災害警報給印度的民眾」,「透過翻譯技術協助烏克蘭人跟波蘭人能夠互相溝通」,「協助彙整全球Covid-19的照護資訊,有超過20億人次使用來找附近診所」,「透過Google手機發送數億條的空襲訊息讓烏克蘭人能夠避難」,這些都是對社會非常有幫助的事情。</p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7Tj77LTdLmmF2bo5QJ2dSEFhyZrW87DUfK8O5PO_9Pci_zKDkB-9O21RApKcHOxymFo96vyFGyz-PsGXONWF1eYj9Qd_dpXsyTdomo6JXfuKM3ZmmM0hKJEZgnIO4NVInrvLJqQpb1kUBijzLkb11VDp5GREE2oaQCUXNTiD4LMMwFms4YGjein8q/s1844/Screen%20Shot%202022-06-09%20at%206.27.10%20PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1118" data-original-width="1844" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7Tj77LTdLmmF2bo5QJ2dSEFhyZrW87DUfK8O5PO_9Pci_zKDkB-9O21RApKcHOxymFo96vyFGyz-PsGXONWF1eYj9Qd_dpXsyTdomo6JXfuKM3ZmmM0hKJEZgnIO4NVInrvLJqQpb1kUBijzLkb11VDp5GREE2oaQCUXNTiD4LMMwFms4YGjein8q/w640-h388/Screen%20Shot%202022-06-09%20at%206.27.10%20PM.png" width="640" /></a></div><br /><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><br /></p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><br /></p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">接下來Jerry老師彙整了Google IO上10件必須要知道的事</p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><o:p> </o:p></p><p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="mso-ansi-language: EN-US;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span lang="EN-US">Google翻譯能夠自主學習新的語言<o:p></o:p></span></p><p align="center" class="MsoListParagraphCxSpMiddle" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm 0cm 0cm 36pt; orphans: auto; text-align: center; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2EOHVHe7oi5OnJZagjc5UzGB6NWrcP0AZnPHfhgpGCAos9zL-xZXDDSWYEovGiGyDPisdGbO4G0BkOTAuN5c2rKCoJ7U6OqjOpap_fo8KlqR52dSIu23fhXm2avgPxAyGvjuub3E0ST6VZjbyDIiyxbLaDkcolkKQJWGCo79T4BNJWOM1NsFsCgVN/s1150/1111.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="718" data-original-width="1150" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2EOHVHe7oi5OnJZagjc5UzGB6NWrcP0AZnPHfhgpGCAos9zL-xZXDDSWYEovGiGyDPisdGbO4G0BkOTAuN5c2rKCoJ7U6OqjOpap_fo8KlqR52dSIu23fhXm2avgPxAyGvjuub3E0ST6VZjbyDIiyxbLaDkcolkKQJWGCo79T4BNJWOM1NsFsCgVN/w640-h400/1111.png" width="640" /></a></div><br /><br /><p></p><p class="MsoListParagraphCxSpMiddle" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm 0cm 0cm 36pt; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><span lang="EN-US">過去學習語言翻譯是採用雙語學習(Bilingual learning),像是英文轉西班牙文,他就要有對應的英文跟西班牙文,但是全世界語言很多,不一定每個語言轉換都有成對的訓練資料可以用。Google的團隊發表了一種做法叫<a href="https://ai.googleblog.com/2022/05/24-new-languages-google-translate.html" style="color: #954f72; text-decoration: underline;">單語學習(Monolingual learning)</a>這種方法可以在Zero-Resource的情況下去學習新的語言翻譯,使用了Transformer-based的模型架構與半監督學習(Semi-supervised)的方式,學習了24種新語言的翻譯,包含阿薩姆語(Assamese)、林加拉語(lingala)、米佐语(Mizo)等。<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm 0cm 0cm 36pt; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><span lang="EN-US"><o:p> </o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm 0cm 0cm 36pt; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><span lang="EN-US">細節請參閱<a href="https://ai.googleblog.com/2022/05/24-new-languages-google-translate.html" target="_blank">文章</a><o:p></o:p></span></p><p class="MsoListParagraphCxSpLast" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm 0cm 0cm 36pt; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><br /></p><p class="MsoNormal" style="-webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; caret-color: rgb(0, 0, 0); color: black; font-family: "Times New Roman", serif; font-size: medium; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; margin: 0cm; orphans: auto; text-align: start; text-decoration: none; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;"><span lang="EN-US"><o:p> </o:p></span></p><p><style class="WebKit-mso-list-quirks-style">
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
mso-themecolor:hyperlink;
text-decoration:underline;
text-underline:single;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:#954F72;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:PMingLiU;
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;
mso-header-margin:35.4pt;
mso-footer-margin:35.4pt;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:887955886;
mso-list-type:hybrid;
mso-list-template-ids:-1854385088 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
-->
</style><span lang="EN-US" style="font-family: "Times New Roman", serif; font-size: 12pt;"><br clear="all" style="break-before: page;" /></span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US" style="font-family: Calibri, sans-serif;"> </span></p><p class="MsoListParagraphCxSpFirst" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google Map建立最佳環保路徑<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUtuXnGaF-XCXej2BN2EtXyLGhtpNEB0m5_ujVwn3OznAcRhao9XKGwfMf-QhZb4nihnh26nvOQ3lmVPUh_gbGMlvMG5UJJwTHzUnxW7mxKcuxO51n6gQMZE8N-igOiv0FSP3TGLjAE-p1vXPo4uFDHayyzhElBck2GJMAaO1Dn_Os9MpfpjEgiv4N/s1150/22222.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="760" data-original-width="1150" height="422" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUtuXnGaF-XCXej2BN2EtXyLGhtpNEB0m5_ujVwn3OznAcRhao9XKGwfMf-QhZb4nihnh26nvOQ3lmVPUh_gbGMlvMG5UJJwTHzUnxW7mxKcuxO51n6gQMZE8N-igOiv0FSP3TGLjAE-p1vXPo4uFDHayyzhElBck2GJMAaO1Dn_Os9MpfpjEgiv4N/w640-h422/22222.png" width="640" /></a></div><br /><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><br /></p><p class="MsoListParagraphCxSpLast" style="font-size: medium;"><span lang="EN-US">Google也透過機器學習的技術,提出了eco-friendly的概念,自動生成最佳的環保路徑提供給使用者使用。</span></p><p class="MsoListParagraphCxSpLast" style="font-size: medium;"><span style="font-family: "PingFang TC", sans-serif;"><br /></span></p><p class="MsoListParagraphCxSpLast" style="font-size: medium;"><span style="font-family: "PingFang TC", sans-serif;">細節請參閱<a href="https://blog.google/products/maps/redefining-what-map-can-be-new-information-and-ai/" target="_blank">文章</a></span></p><p class="MsoListParagraphCxSpLast" style="font-size: medium;"><br /></p><p class="MsoNormal" style="font-size: medium;"><o:p></o:p></p><p class="MsoListParagraphCxSpFirst" style="font-size: medium;"><o:p> </o:p></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US">3.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Youtube能夠提供自動章節分段功能<o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLV81y7cmNqGd0rsWa6tR1aXZc9KceV3DkJJH4mE1ssH24etvQvPGQPVR8ooSBG-ok69l9HqW2tzoyoEZTxX_Xd83f_ueARcF8l2pIUvc9VM3h4QXJYZf5hw4f4BJ0Np8JAIc5C3GHHy8D3FLWb1VUjMGHx-GTjDw44PjWQ0MmjIlaCw94CTlPTiD4/s1241/3333333.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="845" data-original-width="1241" height="436" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLV81y7cmNqGd0rsWa6tR1aXZc9KceV3DkJJH4mE1ssH24etvQvPGQPVR8ooSBG-ok69l9HqW2tzoyoEZTxX_Xd83f_ueARcF8l2pIUvc9VM3h4QXJYZf5hw4f4BJ0Np8JAIc5C3GHHy8D3FLWb1VUjMGHx-GTjDw44PjWQ0MmjIlaCw94CTlPTiD4/w640-h436/3333333.png" width="640" /></a></div><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><br /></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US">能夠自動透過機器學習的方式去分析影片中的內容,進行自動分段的工作,這個技術解決了影片過長,但是使用者想分段去觀賞的問題,同時也可以提升搜尋的效率與結果。<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US">4.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google Meet自動加亮<o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjg4hVrd19mzgVEWTAS6vf1fcENEkackwlR5rPOPJs4tN-BqrmUB6JddQJ2UigOysdRh5Eb-N77kIyvbxufGepv7tQdscwxDLvcsuSbZDHONJOuVgZGTjOHrtahmFWWLkYiaPBG2D3r17jOczzAnx0Ds-6V9DfOW38XTQDz0Yi0JxxUVD5Nn2WTpbRO/s1316/55555.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="739" data-original-width="1316" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjg4hVrd19mzgVEWTAS6vf1fcENEkackwlR5rPOPJs4tN-BqrmUB6JddQJ2UigOysdRh5Eb-N77kIyvbxufGepv7tQdscwxDLvcsuSbZDHONJOuVgZGTjOHrtahmFWWLkYiaPBG2D3r17jOczzAnx0Ds-6V9DfOW38XTQDz0Yi0JxxUVD5Nn2WTpbRO/w640-h360/55555.png" width="640" /></a></div><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US">在Google Meet上 也透過機器學習的方式去做了畫面加亮、去背、增強背景等功能<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"><br /></span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US">5.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google Doc 自動摘要功能<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidMUzL3Vh5ZRqgyX3G6rdlyNud5oH0x9RcjvvdDowihLf-3fX6_u48G6yJcJqnthVb-5VrEQma-DWTOUz8iFIhwQQDfgAQhQ7xGigqiec46EwavS9nRDVEKdREZLWWVhR21nCkPlnNgie8t3i68JARqESpfTJbMBbpuj_bpc4iCZbLXASz-v9Q3ChM/s1308/44444.png" imageanchor="1" style="font-family: -webkit-standard; margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="791" data-original-width="1308" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidMUzL3Vh5ZRqgyX3G6rdlyNud5oH0x9RcjvvdDowihLf-3fX6_u48G6yJcJqnthVb-5VrEQma-DWTOUz8iFIhwQQDfgAQhQ7xGigqiec46EwavS9nRDVEKdREZLWWVhR21nCkPlnNgie8t3i68JARqESpfTJbMBbpuj_bpc4iCZbLXASz-v9Q3ChM/w640-h388/44444.png" width="640" /></a></p><p class="MsoListParagraphCxSpLast" style="font-size: medium;"><span lang="EN-US">這是一個很特別的機器學習技術,在影片中可以看到包含了自然語言理解(NLU)、資訊壓縮(IC)、自然語言生成(NLG)等技術</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">,可以讓</span><span lang="EN-US">Google文件</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">能夠有自動摘要的功能,方便使用者能夠先閱讀摘要掌握文本資訊。當然主要先從英文的會議記錄開始,但是可以想得到未來大量的文本閱讀上,在</span><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">文件上面可以變得非常方便。</span><span style="font-family: "PingFang TC", sans-serif;"> </span></p><p class="MsoListParagraphCxSpFirst" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US">6.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google搜尋可以用照片作為搜尋關鍵字<o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPaR5gOXU3rWWqAo6-wA75evSuoW_hvwXIb20CtSKuz-nqB0KSTfm2ks3Rb2WMrLcZKi4UnCno-mNxaCig2DJmUg6Zyck3QTsnKfvdpPHetANnL-g5w0U7t_VDjK-dqYrBWRlLRTHaODUM1p45MD7KwuF1rVxpUiNvxFTpMhOj0qQNU_kdFd1R90sz/s1129/666666.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="783" data-original-width="1129" height="444" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPaR5gOXU3rWWqAo6-wA75evSuoW_hvwXIb20CtSKuz-nqB0KSTfm2ks3Rb2WMrLcZKi4UnCno-mNxaCig2DJmUg6Zyck3QTsnKfvdpPHetANnL-g5w0U7t_VDjK-dqYrBWRlLRTHaODUM1p45MD7KwuF1rVxpUiNvxFTpMhOj0qQNU_kdFd1R90sz/w640-h444/666666.png" width="640" /></a></div><br /><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><br /></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US">混合搜尋(Multi search)是一個重要的技術,突破了過去搜尋只能用文字處理,過去我們曾用過以圖搜尋、以文搜圖的方式去獲取資訊,但是今年Google搜尋將會提供用圖+文字的混合搜尋的方式,可以把圖片當作文字的一部分,搭配文字進行搜尋。<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US">7.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google助理可以透過擬人化的方式喚醒與互動<o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEkqhsIBPE_DEsZoYHKAucZ0fAutyNLo0S_gEx-qkdlCdyHgjs_TXymFZfvg_4XPYyGObMV8owoHkyK0rSH9SRkjjNsHTe2p2U1j__0kHZjGDnJiiP5UTzP-jZKjePJ-jGlH_NuEHrupN4WY2AEqYoG-tPZ5JSfQ5vgpLACRlj9LpCaIRzyQyjo-nU/s1350/777777.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="758" data-original-width="1350" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEkqhsIBPE_DEsZoYHKAucZ0fAutyNLo0S_gEx-qkdlCdyHgjs_TXymFZfvg_4XPYyGObMV8owoHkyK0rSH9SRkjjNsHTe2p2U1j__0kHZjGDnJiiP5UTzP-jZKjePJ-jGlH_NuEHrupN4WY2AEqYoG-tPZ5JSfQ5vgpLACRlj9LpCaIRzyQyjo-nU/w640-h360/777777.png" width="640" /></a></div><br /><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><br /></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US">Google助理變得更擬人化了,過去我們使用音箱上面會常常會不小心喚醒音箱服務,但是現在音箱多了視覺辨識功能,我們可以用「Look and Talk」的功能,用看然後再說話的方式,喚醒音箱,想像就是對人講話的方式來做互動。<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US">詳細請參閱<a href="https://www.youtube.com/watch?v=nP-nMZpLM1A&t=2082s" target="_blank">影片</a><o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><br /></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]-->8.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span><!--[endif]--><span lang="EN-US">更強大的自然語言理解模型</span>PaLM</p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6N1TcY7DrYljrGFRaV8dPJLfYU2XKGDBrs7tcexuDxmT811xIvnDRkoATUzSVtF65lXVSwfbe60-8QxZrGyCgFOHK8MpLm81X9M6h3rFoJH3c-nPrmkATQiss-BtRHEPJSU5eIxw_wFekpigwXqQVQdIuNWX_t2o7Ng8MzNI_UzgHkCtLnlZzrlIb/s1326/Screen%20Shot%202022-06-09%20at%206.33.14%20PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="488" data-original-width="1326" height="236" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6N1TcY7DrYljrGFRaV8dPJLfYU2XKGDBrs7tcexuDxmT811xIvnDRkoATUzSVtF65lXVSwfbe60-8QxZrGyCgFOHK8MpLm81X9M6h3rFoJH3c-nPrmkATQiss-BtRHEPJSU5eIxw_wFekpigwXqQVQdIuNWX_t2o7Ng8MzNI_UzgHkCtLnlZzrlIb/w640-h236/Screen%20Shot%202022-06-09%20at%206.33.14%20PM.png" width="640" /></a></div><br /><p></p><p class="MsoListParagraphCxSpLast" style="font-size: medium;">去年<span lang="EN-US">Google才出了LaMDA</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">,今年又出了一個</span><span lang="EN-US" style="font-family: "PingFang TC", sans-serif;">PaLM</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">,事實上</span><span lang="EN-US" style="font-family: "PingFang TC", sans-serif;">Google</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">希望能夠打造出跨語言的自然語言理解模型,這個</span><span lang="EN-US" style="font-family: "PingFang TC", sans-serif;">PaLM</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">能夠處理的自然語言理解任務又更多了,</span><span lang="EN-US" style="font-family: "PingFang TC", sans-serif;">Jerry</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">老師將會再另外找時間去做比較深度的分享,基本上應該會出現在</span><span lang="EN-US" style="font-family: "PingFang TC", sans-serif;">Google Devfest</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">的活動上,敬請期待!</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoListParagraphCxSpLast" style="font-size: medium;"><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;"><br /></span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US">9.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">手機</span>自動檢測車禍、自動撥打緊急電話<span lang="EN-US"><o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvsFunpJ5fsIIkIXVltMDzLWsYorI7MQvhw0H0NqLdxKXy4f0vBh_4f43VwzRHzeOtZBdIPJka8Z1rjpm_e0KBP_HyVUcfbPzQ97xCXfBjSwFtrOiRvQNUXvRC1XgcqI6RxUHs7chE3R6XBW78D58NwqcWq5R0RFX09qeZoSWDiQp1EmH6bniz4Kbs/s1212/10101010.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="760" data-original-width="1212" height="402" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvsFunpJ5fsIIkIXVltMDzLWsYorI7MQvhw0H0NqLdxKXy4f0vBh_4f43VwzRHzeOtZBdIPJka8Z1rjpm_e0KBP_HyVUcfbPzQ97xCXfBjSwFtrOiRvQNUXvRC1XgcqI6RxUHs7chE3R6XBW78D58NwqcWq5R0RFX09qeZoSWDiQp1EmH6bniz4Kbs/w640-h402/10101010.png" width="640" /></a></div><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><br /></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">在影片中介紹不少</span><span lang="EN-US" style="font-family: "PingFang TC", sans-serif;">Google</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">手機、手錶應用,讓我印象深刻的是自動檢測車禍、自動撥打電話功能,請參閱影片來感受一下這個技術對於人們有多大的幫助</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">詳細請參閱<a href="https://youtu.be/nP-nMZpLM1A?t=4653" target="_blank">影片</a></span></p><br /><p class="MsoListParagraphCxSpLast" style="font-size: medium;"> </p><p class="MsoListParagraphCxSpFirst" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US">10.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google眼鏡與翻譯的結合<o:p></o:p></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimHNpn_Z14u2yPCpQBW8oakoX19PTCSSkfMAxJ13nPW8MovWETlbgckBbvRwR_dN60rD7AXySqd_NaeWJob6hfAf_9EKMV3vU49a9XilpspVI4RGC2ArqReaNJZgnbA4_MU4lcF04Snf6oNaXMClQxZrtl66CvbJHnDMTADQq8OPZOo5DzfWBiAwDz/s1262/12121212.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="712" data-original-width="1262" height="362" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimHNpn_Z14u2yPCpQBW8oakoX19PTCSSkfMAxJ13nPW8MovWETlbgckBbvRwR_dN60rD7AXySqd_NaeWJob6hfAf_9EKMV3vU49a9XilpspVI4RGC2ArqReaNJZgnbA4_MU4lcF04Snf6oNaXMClQxZrtl66CvbJHnDMTADQq8OPZOo5DzfWBiAwDz/w640-h362/12121212.png" width="640" /></a></div><br /><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><br /></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">最後是</span><span lang="EN-US" style="font-family: "PingFang TC", sans-serif;">Google</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">測試中的產品,目標也是在希望能夠讓人與人溝通能夠更加便利,透過這個</span><span lang="EN-US" style="font-family: "PingFang TC", sans-serif;">Google</span><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">眼鏡,能夠做快速翻譯的動作,透過分析雙方的聲音,能夠直接在眼鏡上用自己習慣的語言做互動,對於語言不通的兩個人,可以做有效率的溝通。</span><span lang="EN-US"><o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;"><br /></span></p><p class="MsoListParagraphCxSpMiddle" style="font-size: medium;"><span lang="ZH-TW" style="font-family: "PingFang TC", sans-serif;">詳細請參閱<a href="https://youtu.be/nP-nMZpLM1A?t=6982" target="_blank">影片</a></span></p><br /><p class="MsoListParagraphCxSpLast" style="font-size: medium;"><o:p> </o:p></p><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;">以上做簡單分享,供大家參考!</div><div class="separator" style="clear: both; text-align: left;"><br /></div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><br />資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.comtag:blogger.com,1999:blog-5898372732399324805.post-74408514933500279722020-10-17T13:29:00.017+08:002020-10-18T09:22:48.489+08:00如何透過TensorFlow建構負責任的AI系統-DevFest 2020<p>今天的devfest2020 <a href="https://devfest2020.web.app/" rel="nofollow" target="_blank">活動</a>非常精彩,感謝大家的參與</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-T3JLFOBA2Jk/X4uYApqAicI/AAAAAAAAVHg/SMs-cGQudfsPcIjP4LXDZ5nmRL9KS9IEgCLcBGAsYHQ/s1880/Screen%2BShot%2B2020-10-18%2Bat%2B9.18.36%2BAM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="634" data-original-width="1880" height="135" src="https://1.bp.blogspot.com/-T3JLFOBA2Jk/X4uYApqAicI/AAAAAAAAVHg/SMs-cGQudfsPcIjP4LXDZ5nmRL9KS9IEgCLcBGAsYHQ/w400-h135/Screen%2BShot%2B2020-10-18%2Bat%2B9.18.36%2BAM.png" width="400" /></a></div><p><br /></p><p>今天主要分享如何透過TensorFlow以及Google相關的技術, 建立可解釋AI(Explainable AI)與具備負責任的AI(Responsible AI),並且打造具備負責任的機器學習機制, 包含從界定問題、建構及準備資料、建構與訓練模型、 評估模型、部署與監控等流程,運用的工具像是 Tensorboard、Embedding Projector、What-If Tool等, 協助大家了解機器學習的近期新發展方向。</p><p>附上活動的簡報與相關連結。</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-F3yP-EO2Ox4/X4qB2PmOX6I/AAAAAAAAVHU/My6J_y6VXzYeb9Cm38DIWsdmeAoBCaZFQCLcBGAsYHQ/s2048/11.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1536" data-original-width="2048" src="https://1.bp.blogspot.com/-F3yP-EO2Ox4/X4qB2PmOX6I/AAAAAAAAVHU/My6J_y6VXzYeb9Cm38DIWsdmeAoBCaZFQCLcBGAsYHQ/s320/11.jpg" width="320" /></a></div><br /><p><br /></p><p>1.<a href="https://github.com/jerrywu2013/Responsible-ai" rel="nofollow" target="_blank">演講簡報</a></p><p>2.<a href="https://hiskio.com/courses/413/about?promo_code=LG282JG" rel="nofollow" target="_blank">機器學習與應用</a></p><p>3.<a href="https://tf.wiki/" rel="nofollow" target="_blank">更多了解TensorFlow</a></p><p>4.<a href="https://www.tensorflow.org/resources/responsible-ai?hl=zh-tw" rel="nofollow" target="_blank">RAI with TensorFlow</a></p><p>5.<a href="https://youtu.be/vA21dx5dPsE?t=5414" rel="nofollow" target="_blank">直播回放網址</a></p>資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-4611116654517136092020-07-26T10:10:00.001+08:002020-07-26T10:10:31.310+08:00機器學習 x 19 種產業應用|快速找出最佳演算法非常興奮能夠跟大家分享Jerry老師的最新線上課程。<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/--Q3jKjB8gSQ/XxzmB2Ji4kI/AAAAAAAAVGA/geMFMByvZRk_INCgn-OZT1TZ_wCpK1u2gCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-07-26%2Bat%2B10.10.01%2BAM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="666" data-original-width="1182" height="360" src="https://1.bp.blogspot.com/--Q3jKjB8gSQ/XxzmB2Ji4kI/AAAAAAAAVGA/geMFMByvZRk_INCgn-OZT1TZ_wCpK1u2gCLcBGAsYHQ/s640/Screen%2BShot%2B2020-07-26%2Bat%2B10.10.01%2BAM.png" width="640" /></a></div>
<br />
<br />
這堂課程叫做「機器學習 x 19 種產業應用|快速找出最佳演算法」,有鑒於大家對於機器學習的落地還充滿很多問題,因此Jerry老師特別把過去的經驗整理成課程,幫助大家了解機器學習如何在19種產業實現。<br />
<br />
不過課程還沒開放之前,有一份問券需要懇請各位讀者幫忙填寫<br />
<a href="https://www.surveycake.com/s/Zq4k7">https://www.surveycake.com/s/Zq4k7</a><br />
<br />
以利Jerry老師強化課程內容<br />
也謝謝大家支持喔!<br />
<br />
<br />
<br />資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-35824344270161037732020-05-15T11:37:00.005+08:002020-05-15T11:45:39.531+08:00五分鐘學會使用TPU訓練BERT自然語言模型<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<br />
大家好我是Jerry老師,這篇帶來了五分鐘學會系列,主要跟大家分享如何在GCP上採用TPU來訓練自然語言模型,整個過程不包含「訓練時間、主機設定與安裝時間」只需要五分鐘就可以幾行指令就可以完成訓練操作。<br />
<div>
<br /></div>
<div>
<a href="https://1.bp.blogspot.com/-RRdAgTT-EXE/Xr4P6ntd-cI/AAAAAAAAVE0/u4kaLEXEg-EXLeBk4Ozqiv4Wkd66ZFt0gCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-05-15%2Bat%2B11.42.14%2BAM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="550" data-original-width="1482" height="236" src="https://1.bp.blogspot.com/-RRdAgTT-EXE/Xr4P6ntd-cI/AAAAAAAAVE0/u4kaLEXEg-EXLeBk4Ozqiv4Wkd66ZFt0gCLcBGAsYHQ/s640/Screen%2BShot%2B2020-05-15%2Bat%2B11.42.14%2BAM.png" width="640" /></a><br />
<br />
首先大家必須要有GCP帳號,只要有申請Gmail,就可以快速開通。目前有300元美金免費試用,所以不用擔心費用,開通請往<a href="https://cloud.google.com/" target="_blank">這裡</a>。</div>
<div>
<br /></div>
<div>
<br />
<br /></div>
<div>
有了主控台之後,就可以開始玩了。</div>
<div>
在主控台畫面,點選「新增專案」</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-5Vh30-XB620/Xr4I6ByvjVI/AAAAAAAAVDY/hFaHDqfdCrcV2CJhFwHsmVq1M5am3HgrwCLcBGAsYHQ/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="993" data-original-width="1600" height="396" src="https://1.bp.blogspot.com/-5Vh30-XB620/Xr4I6ByvjVI/AAAAAAAAVDY/hFaHDqfdCrcV2CJhFwHsmVq1M5am3HgrwCLcBGAsYHQ/s640/1.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著可以會有一個自動產生的專案名稱,我們可以自己輸入,但是要記得輸入的名稱。</div>
<div class="separator" style="clear: both; text-align: left;">
像是我在自動產生的名稱前後加上jerry跟tpu。</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-dv8x3XdLtug/Xr4JJdRyPTI/AAAAAAAAVDc/e71RAX3_zbE30sk9kNGcoNVaVsX_i1a9gCLcBGAsYHQ/s1600/2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1064" data-original-width="1266" height="536" src="https://1.bp.blogspot.com/-dv8x3XdLtug/Xr4JJdRyPTI/AAAAAAAAVDc/e71RAX3_zbE30sk9kNGcoNVaVsX_i1a9gCLcBGAsYHQ/s640/2.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著回主控台,在畫面右上角,開啟「Cloud Shell」</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-5H-WtywGWbs/Xr4Jd7hL1vI/AAAAAAAAVDo/_X5mUqjDO2MxyR3YZhiPktmrsY926PF-wCLcBGAsYHQ/s1600/3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="790" data-original-width="1600" height="316" src="https://1.bp.blogspot.com/-5H-WtywGWbs/Xr4Jd7hL1vI/AAAAAAAAVDo/_X5mUqjDO2MxyR3YZhiPktmrsY926PF-wCLcBGAsYHQ/s640/3.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
1.接著輸入以下參數</div>
<div class="separator" style="clear: both; text-align: left;">
export PROJECT_NAME=<span style="color: red;">jerry-project87630-tpu</span>(注意這是剛剛我設定的ID)</div>
<div class="separator" style="clear: both; text-align: left;">
請在紅字修改自己剛剛開的專案名稱</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
2.執行下面指令</div>
<div class="separator" style="clear: both; text-align: left;">
gcloud config set project ${PROJECT_NAME}</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
3.建立bucket-name,這是Google專門存放資料的技術,我取的跟ID一樣好了</div>
<div class="separator" style="clear: both; text-align: left;">
gsutil mb -p ${PROJECT_NAME} -c standard -l europe-west4 -b on gs://<span style="color: red;">jerry-project87630-tpu</span></div>
<div class="separator" style="clear: both; text-align: left;">
一樣請在紅字修改自己剛剛開的專案名稱</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-Z1vLy1aKihc/Xr4JytJYOuI/AAAAAAAAVDw/hgS5Ui2YyAokTZATd-sYySFMah3uu2uAwCLcBGAsYHQ/s1600/4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="181" data-original-width="1600" height="72" src="https://1.bp.blogspot.com/-Z1vLy1aKihc/Xr4JytJYOuI/AAAAAAAAVDw/hgS5Ui2YyAokTZATd-sYySFMah3uu2uAwCLcBGAsYHQ/s640/4.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著就正式開啟TPU囉</div>
<div class="separator" style="clear: both; text-align: left;">
請輸入下面指令,這邊我的TPU名稱設定jerry,大家照貼即可。</div>
<div class="separator" style="clear: both; text-align: left;">
ctpu up --tpu-size=v3-8 --name=jerry --machine-type=n1-standard-8 --zone=europe-west4-a --tf-version=2.2</div>
<div class="separator" style="clear: both; text-align: left;">
這邊採用的是最新TensorFlow2.2版本</div>
<div class="separator" style="clear: both; text-align: left;">
看到下面畫面,打 y 即可</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-rdEqyFy1kx4/Xr4Kn1BPrUI/AAAAAAAAVD8/6i5AgiIoboIbJlyNDcPgyZmhHypBKl3ygCLcBGAsYHQ/s1600/5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="860" data-original-width="1382" height="398" src="https://1.bp.blogspot.com/-rdEqyFy1kx4/Xr4Kn1BPrUI/AAAAAAAAVD8/6i5AgiIoboIbJlyNDcPgyZmhHypBKl3ygCLcBGAsYHQ/s640/5.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著會看到下面畫面,這是TPU建立過程。</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-AzvONz9pPIw/Xr4KyrosA4I/AAAAAAAAVEA/a4rxeQ0r1K4ABM16j7YfffmWW3rhTwtlQCLcBGAsYHQ/s1600/6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="894" data-original-width="1456" height="392" src="https://1.bp.blogspot.com/-AzvONz9pPIw/Xr4KyrosA4I/AAAAAAAAVEA/a4rxeQ0r1K4ABM16j7YfffmWW3rhTwtlQCLcBGAsYHQ/s640/6.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著進入我們剛剛設定好的TPU機器,執行下面指令</div>
<div class="separator" style="clear: both; text-align: left;">
gcloud compute ssh jerry --zone=europe-west4-a</div>
<div class="separator" style="clear: both; text-align: left;">
中間如果看到以下畫面,看到這幾行直接按Enter即可。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-UFSuj3bAaTk/Xr4K-lmEPWI/AAAAAAAAVEI/ZgYuISVg4zUNbbRThwdEpXyvQs091gDegCLcBGAsYHQ/s1600/7.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="496" data-original-width="1342" height="236" src="https://1.bp.blogspot.com/-UFSuj3bAaTk/Xr4K-lmEPWI/AAAAAAAAVEI/ZgYuISVg4zUNbbRThwdEpXyvQs091gDegCLcBGAsYHQ/s640/7.png" width="640" /></a></div>
<div class="separator" style="clear: both;">
接著設定環境參數,執行下面指令</div>
<div class="separator" style="clear: both;">
export TPU_NAME=jerry</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
設定BUCKET位置,這個跟前面設定一樣</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
export STORAGE_BUCKET=gs://<span style="color: red;">jerry-project87630-tpu</span></div>
<div class="separator" style="clear: both;">
一樣請在紅字修改自己剛剛開的專案名稱</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
然後安裝執行這次實作的套件</div>
<pre class="lang-sh tat-client-setup" dir="ltr" is-upgraded="" style="background-color: #f1f3f4; box-sizing: inherit; color: #37474f; font-family: "Roboto Mono", monospace; font-size: 14px; font-stretch: normal; line-height: 20px; orphans: 2; overflow-x: auto; padding: 24px; position: relative; widows: 2;" translate="no"><span class="pln" style="box-sizing: inherit;">sudo pip3 install </span><span class="pun" style="box-sizing: inherit;">-</span><span class="pln" style="box-sizing: inherit;">r </span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">usr</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">share</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">models</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">official</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">requirements</span><span class="pun" style="box-sizing: inherit;">.</span><span class="pln" style="box-sizing: inherit;">txt</span></pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-stBgPDRUGes/Xr4LtSv_IsI/AAAAAAAAVEU/2vzMT499LTob49yKfbcUAOvFpx5nzVzxACLcBGAsYHQ/s1600/8.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1062" data-original-width="1500" height="452" src="https://1.bp.blogspot.com/-stBgPDRUGes/Xr4LtSv_IsI/AAAAAAAAVEU/2vzMT499LTob49yKfbcUAOvFpx5nzVzxACLcBGAsYHQ/s640/8.png" width="640" /></a></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
最後就剩兩個步驟,設定以下環境參數</div>
<div class="separator" style="clear: both;">
<br /></div>
<pre class="lang-sh tat-client-exports" dir="ltr" is-upgraded="" style="background-color: #f1f3f4; box-sizing: inherit; color: #37474f; font-family: "Roboto Mono", monospace; font-size: 14px; font-stretch: normal; line-height: 20px; orphans: 2; overflow-x: auto; padding: 24px; position: relative; widows: 2;" translate="no"><span class="pln" style="box-sizing: inherit;">export PYTHONPATH</span><span class="pun" style="box-sizing: inherit;">=</span><span class="str" style="box-sizing: inherit; color: #0d904f;">"${PYTHONPATH}:/usr/share/models"</span><span class="pln" style="box-sizing: inherit;">
</span><span class="no-select" style="box-sizing: inherit;"><span class="pun" style="box-sizing: inherit;">(</span><span class="pln" style="box-sizing: inherit;">vm</span><span class="pun" style="box-sizing: inherit;">)</span><span class="pln" style="box-sizing: inherit;">$ </span></span><span class="pln" style="box-sizing: inherit;">export BERT_BASE_DIR</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">gs</span><span class="pun" style="box-sizing: inherit;">://</span><span class="pln" style="box-sizing: inherit;">cloud</span><span class="pun" style="box-sizing: inherit;">-</span><span class="pln" style="box-sizing: inherit;">tpu</span><span class="pun" style="box-sizing: inherit;">-</span><span class="pln" style="box-sizing: inherit;">checkpoints</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">bert</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">keras_bert</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">uncased_L</span><span class="pun" style="box-sizing: inherit;">-</span><span class="lit" style="box-sizing: inherit; color: #c53929;">24</span><span class="pln" style="box-sizing: inherit;">_H</span><span class="pun" style="box-sizing: inherit;">-</span><span class="lit" style="box-sizing: inherit; color: #c53929;">1024</span><span class="pln" style="box-sizing: inherit;">_A</span><span class="pun" style="box-sizing: inherit;">-</span><span class="lit" style="box-sizing: inherit; color: #c53929;">16</span><span class="pln" style="box-sizing: inherit;">
</span><span class="no-select" style="box-sizing: inherit;"><span class="pun" style="box-sizing: inherit;">(</span><span class="pln" style="box-sizing: inherit;">vm</span><span class="pun" style="box-sizing: inherit;">)</span><span class="pln" style="box-sizing: inherit;">$ </span></span><span class="pln" style="box-sizing: inherit;">export MODEL_DIR</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">STORAGE_BUCKET</span><span class="pun" style="box-sizing: inherit;">}/</span><span class="pln" style="box-sizing: inherit;">bert</span><span class="pun" style="box-sizing: inherit;">-</span><span class="pln" style="box-sizing: inherit;">output</span><span class="no-select" style="box-sizing: inherit;"><span class="pun" style="box-sizing: inherit;">(</span><span class="pln" style="box-sizing: inherit;">vm</span><span class="pun" style="box-sizing: inherit;">)</span><span class="pln" style="box-sizing: inherit;">$ </span></span><span class="pln" style="box-sizing: inherit;">export GLUE_DIR</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">gs</span><span class="pun" style="box-sizing: inherit;">://</span><span class="pln" style="box-sizing: inherit;">cloud</span><span class="pun" style="box-sizing: inherit;">-</span><span class="pln" style="box-sizing: inherit;">tpu</span><span class="pun" style="box-sizing: inherit;">-</span><span class="pln" style="box-sizing: inherit;">checkpoints</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">bert</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">classification</span><span class="no-select" style="box-sizing: inherit;"><span class="pun" style="box-sizing: inherit;">(</span><span class="pln" style="box-sizing: inherit;">vm</span><span class="pun" style="box-sizing: inherit;">)</span><span class="pln" style="box-sizing: inherit;">$ </span></span><span class="pln" style="box-sizing: inherit;">export TASK</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">mnli</span></pre>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
接著跑模型,執行下面指令</div>
<div class="separator" style="clear: both;">
<br /></div>
<pre class="lang-sh test-required tat-train-donuts-functional" dir="ltr" is-upgraded="" style="background-color: #f1f3f4; box-sizing: inherit; color: #37474f; font-family: "Roboto Mono", monospace; font-size: 14px; font-stretch: normal; line-height: 20px; orphans: 2; overflow-x: auto; padding: 24px; position: relative; widows: 2;" translate="no"><span class="pln" style="box-sizing: inherit;">python3 </span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">usr</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">share</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">models</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">official</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">nlp</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">bert</span><span class="pun" style="box-sizing: inherit;">/</span><span class="pln" style="box-sizing: inherit;">run_classifier</span><span class="pun" style="box-sizing: inherit;">.</span><span class="pln" style="box-sizing: inherit;">py </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">mode</span><span class="pun" style="box-sizing: inherit;">=</span><span class="str" style="box-sizing: inherit; color: #0d904f;">'train_and_eval'</span><span class="pln" style="box-sizing: inherit;"> </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">input_meta_data_path</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">GLUE_DIR</span><span class="pun" style="box-sizing: inherit;">}/</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">TASK</span><span class="pun" style="box-sizing: inherit;">}</span><span class="pln" style="box-sizing: inherit;">_meta_data </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">train_data_path</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">GLUE_DIR</span><span class="pun" style="box-sizing: inherit;">}/</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">TASK</span><span class="pun" style="box-sizing: inherit;">}</span><span class="pln" style="box-sizing: inherit;">_train</span><span class="pun" style="box-sizing: inherit;">.</span><span class="pln" style="box-sizing: inherit;">tf_record </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">eval_data_path</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">GLUE_DIR</span><span class="pun" style="box-sizing: inherit;">}/</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">TASK</span><span class="pun" style="box-sizing: inherit;">}</span><span class="pln" style="box-sizing: inherit;">_eval</span><span class="pun" style="box-sizing: inherit;">.</span><span class="pln" style="box-sizing: inherit;">tf_record </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">bert_config_file</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">BERT_BASE_DIR</span><span class="pun" style="box-sizing: inherit;">}/</span><span class="pln" style="box-sizing: inherit;">bert_config</span><span class="pun" style="box-sizing: inherit;">.</span><span class="pln" style="box-sizing: inherit;">json </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">init_checkpoint</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">BERT_BASE_DIR</span><span class="pun" style="box-sizing: inherit;">}/</span><span class="pln" style="box-sizing: inherit;">bert_model</span><span class="pun" style="box-sizing: inherit;">.</span><span class="pln" style="box-sizing: inherit;">ckpt </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">train_batch_size</span><span class="pun" style="box-sizing: inherit;">=</span><span class="lit" style="box-sizing: inherit; color: #c53929;">32</span><span class="pln" style="box-sizing: inherit;"> </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">eval_batch_size</span><span class="pun" style="box-sizing: inherit;">=</span><span class="lit" style="box-sizing: inherit; color: #c53929;">32</span><span class="pln" style="box-sizing: inherit;"> </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">learning_rate</span><span class="pun" style="box-sizing: inherit;">=</span><span class="lit" style="box-sizing: inherit; color: #c53929;">2e-5</span><span class="pln" style="box-sizing: inherit;"> </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">num_train_epochs</span><span class="pun" style="box-sizing: inherit;">=</span><span class="lit" style="box-sizing: inherit; color: #c53929;">3</span><span class="pln" style="box-sizing: inherit;"> </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">model_dir</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">MODEL_DIR</span><span class="pun" style="box-sizing: inherit;">}</span><span class="pln" style="box-sizing: inherit;"> </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">distribution_strategy</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">tpu </span><span class="pun" style="box-sizing: inherit;">\</span><span class="pln" style="box-sizing: inherit;">
</span><span class="pun" style="box-sizing: inherit;">--</span><span class="pln" style="box-sizing: inherit;">tpu</span><span class="pun" style="box-sizing: inherit;">=</span><span class="pln" style="box-sizing: inherit;">$</span><span class="pun" style="box-sizing: inherit;">{</span><span class="pln" style="box-sizing: inherit;">TPU_NAME</span><span class="pun" style="box-sizing: inherit;">}</span></pre>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
看到下面畫面,表示跑成功囉,大約一小時之後會有結果,中途如果要中斷,就按Control+z</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-NGv53QUrkMw/Xr4Mdfo2D2I/AAAAAAAAVEc/Js2-_0gLwDoEkc092rls1OUds3TXyCUWwCLcBGAsYHQ/s1600/9.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="779" data-original-width="1600" height="310" src="https://1.bp.blogspot.com/-NGv53QUrkMw/Xr4Mdfo2D2I/AAAAAAAAVEc/Js2-_0gLwDoEkc092rls1OUds3TXyCUWwCLcBGAsYHQ/s640/9.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
因為TPU非常貴,所以下面就是關閉的程序</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
首先離開VM</div>
<pre class="lang-sh tat-skip" dir="ltr" is-upgraded="" style="background-color: #f1f3f4; box-sizing: inherit; color: #37474f; font-family: "Roboto Mono", monospace; font-size: 14px; font-stretch: normal; line-height: 20px; orphans: 2; overflow-x: auto; padding: 24px; position: relative; widows: 2;" translate="no"><span class="pln" style="box-sizing: inherit;">exit</span></pre>
<div class="separator" style="clear: both;">
<br /></div>
<div>
接著確認目前的TPU有哪些還在執行</div>
<div>
<pre class="lang-sh tat-skip" dir="ltr" is-upgraded="" style="background-color: #f1f3f4; box-sizing: inherit; color: #37474f; font-family: "Roboto Mono", monospace; font-size: 14px; font-stretch: normal; line-height: 20px; orphans: 2; overflow-x: auto; padding: 24px; position: relative; widows: 2;" translate="no">ctpu list</pre>
</div>
<div>
<span class="pln" style="box-sizing: inherit;"><br /></span></div>
<div>
接著刪除指定的TPU,剛剛我設定的TPU名字是jerry,在下面畫面按 y 就可以刪除TPU囉!</div>
<div>
<br /></div>
<div>
<div>
<pre class="lang-sh tat-skip" dir="ltr" is-upgraded="" style="background-color: #f1f3f4; box-sizing: inherit; color: #37474f; font-family: "Roboto Mono", monospace; font-size: 14px; font-stretch: normal; line-height: 20px; orphans: 2; overflow-x: auto; padding: 24px; position: relative; widows: 2;" translate="no">ctpu delete jerry</pre>
</div>
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-2L_BhYAYUDQ/Xr4NRwQYm4I/AAAAAAAAVEo/lQaEBzVL6GgtqEc6xse_vINAAlWHkZc5ACLcBGAsYHQ/s1600/Screen%2BShot%2B2020-05-15%2Bat%2B10.54.01%2BAM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="304" data-original-width="1076" height="180" src="https://1.bp.blogspot.com/-2L_BhYAYUDQ/Xr4NRwQYm4I/AAAAAAAAVEo/lQaEBzVL6GgtqEc6xse_vINAAlWHkZc5ACLcBGAsYHQ/s640/Screen%2BShot%2B2020-05-15%2Bat%2B10.54.01%2BAM.png" width="640" /></a></div>
<div>
<br /></div>
<div>
<div>
<br /></div>
<div>
目前的Google TPU費用,不同區域與等級都有不同價格</div>
<div>
平均一個小時1塊到9塊美金不等</div>
<div>
<br /></div>
<div>
目前如果採用標準TPU-V2 約5塊美金,約156元台幣,跑一天差不多3800左右。</div>
<div>
跑一個月就是11萬多,如果用好一點TPU-V3 約8塊美金,跑一個月17萬。</div>
<div>
感謝文章的TPU費用是由Google雲端贊助的費用(GCP credit supported activity)。</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
以上是這篇文章跟大家分享透過TPU如何跑BERT模型,以及費用成本估算,感謝收看。</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-27498779692716534712020-03-30T12:18:00.001+08:002020-03-30T12:19:30.523+08:00五分鐘掌握2020的TensorFlow新發展-機器學習公平性指標篇<br />
這次的<a href="https://youtu.be/HlBGYxO8RaU" target="_blank">TensorFlow的2020年研討會</a>還有正式公開的工具叫做<a href="https://www.ithome.com.tw/news/135710" target="_blank">公平性衡量指標(Fairness Indicators)</a>,這個是非常進階的機器學習工具。而公平性衡量指標(Fairness Indicators)是近年機器學習有關公平性研究熱門的研究領域,主要用於二元、多分類的機器學習題目。<br />
<br />
過去的評估工具在大量數據集、大量的模型的情境下處理都不太好,Google需要十億個客戶系統上運作一個穩定的機器學習模型評估工具就變得非常重要。該工具評估的項目會包含:數據的分佈、模型的性能、多個不同參數下的表現。<br />
<br />
<br />
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="315" src="https://www.youtube.com/embed/pHT-ImFXPQo" width="560"></iframe><br />
<br />
上述的影片就是Google介紹如何透過公平性衡量指標(Fairness Indicators)概念實現公平的機器學習。<br />
<br />
ML Practicum: Fairness in Perspective API相關的課程<br />
<a href="https://developers.google.com/machine-learning/practica/fairness-indicators">https://developers.google.com/machine-learning/practica/fairness-indicators</a><br />
<br />
在也指出幾個很重要的機器學習問題<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-yqlcPf8C4sI/XoFyNkj-3uI/AAAAAAAAVCk/mwutlJy0xjwaQ-50QQiIChAy17_T-BVoQCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-30%2Bat%2B12.07.03%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="654" data-original-width="1270" height="328" src="https://1.bp.blogspot.com/-yqlcPf8C4sI/XoFyNkj-3uI/AAAAAAAAVCk/mwutlJy0xjwaQ-50QQiIChAy17_T-BVoQCLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-30%2Bat%2B12.07.03%2BPM.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
從上圖上來觀察可以看到機器學習在各個流程上</div>
<div class="separator" style="clear: both;">
包含從定義問題、準備資料、建立模型、部署、迭代的過程</div>
<div class="separator" style="clear: both;">
都會出現人的主觀行為,進而影響到學習的偏差。</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
因此Google提出了一個機器學習公平性研究的工作流程</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-Netg5ooSnzY/XoFyWhAS4HI/AAAAAAAAVCo/-ZvaCLTyKDkAtT23XlJTNDd0cQ3h96AgwCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-30%2Bat%2B12.10.40%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="903" data-original-width="1600" height="360" src="https://1.bp.blogspot.com/-Netg5ooSnzY/XoFyWhAS4HI/AAAAAAAAVCo/-ZvaCLTyKDkAtT23XlJTNDd0cQ3h96AgwCLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-30%2Bat%2B12.10.40%2BPM.png" width="640" /></a></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
包含透過Tensorflow的資料驗證工具、模型分析工具、互動式視覺化工具來達到</div>
<div class="separator" style="clear: both;">
改善機器學習有人類偏差的問題。</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
在這些介紹後,大家一定很好奇工具會有哪些。</div>
<div class="separator" style="clear: both;">
首先這個必須學會<a href="https://www.tensorflow.org/tensorboard/get_started" target="_blank">TensorBoard</a>,這個工具以前主要是用來視覺化模型的表現,在2.0之後功能就變得非常的多元。理解了TensorBoard(TensorFlow視覺化)之後,就可以理解以下的工具:</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
Tensorflow Data Analysis (TFDV):主要用於分析數據集的分佈。</div>
<div class="separator" style="clear: both;">
Tensorflow Model Analysis (TFMA):主要用於分析模型的效能、也包含公平性指標可以分析不同模型表現。</div>
<div class="separator" style="clear: both;">
The What-If Tool (WIT):主要透過互動的方式分析模型。</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
光說明沒有範例不夠看,所以請透過下面連結,可以直接到工具的範例中去深讀內容。</div>
<div class="separator" style="clear: both;">
Tensorflow Data Analysis (TFDV)</div>
<div class="separator" style="clear: both;">
<a href="https://www.tensorflow.org/tfx/guide/tfdv">https://www.tensorflow.org/tfx/guide/tfdv</a></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
Tensorflow Model Analysis (TFMA)</div>
<div class="separator" style="clear: both;">
<a href="https://www.tensorflow.org/tfx/model_analysis/get_started">https://www.tensorflow.org/tfx/model_analysis/get_started</a></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
What-If Tool Dashboard</div>
<div class="separator" style="clear: both;">
<a href="https://www.tensorflow.org/tensorboard/what_if_tool">https://www.tensorflow.org/tensorboard/what_if_tool</a></div>
<div>
<br /></div>
以上就是機器學習關於TensorFlow在公平性指標的簡單介紹。資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com1tag:blogger.com,1999:blog-5898372732399324805.post-65420725199994449682020-03-20T20:06:00.000+08:002020-03-20T20:16:01.496+08:00五分鐘掌握2020的TensorFlow新發展-TF_Hub模型篇想要免費的預訓練模型(Pre-trained Model)或演算法嗎?這裡通通都有。<br />
<br />
這次的TensorFlow研討會還有正式公開的工具叫做<a href="https://tfhub.dev/" rel="nofollow" target="_blank">TensorFlow Hub</a>,裡面目前為止累積<span style="color: purple;"> </span><b><span style="color: purple;"><u>472</u></span> </b>個(2020/03)模型可以使用,大致上分為圖片(Image)、文字(Text)、影片(Video)<br />
三種資料類型的模型,而且在裝置上面支持基本的TensorFlow之外,也包含TF.js、TFLite、Coral。<br />
<br />
首先我們可以進入這裡 <a href="https://tfhub.dev/">https://tfhub.dev/</a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-0Id8qgk2LZo/XnSSaZlMjEI/AAAAAAAAVBw/O5xvCPk4wyImG8xxnzvXb4g2tK2E4BijwCLcBGAsYHQ/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="826" data-original-width="1600" height="329" src="https://1.bp.blogspot.com/-0Id8qgk2LZo/XnSSaZlMjEI/AAAAAAAAVBw/O5xvCPk4wyImG8xxnzvXb4g2tK2E4BijwCLcBGAsYHQ/s640/1.png" width="640" /></a></div>
<br />
<br />
接著就可以根據自己需求挑選模型,譬如說筆者這兩年流行的BERT,挑選<a href="https://tfhub.dev/google/bert_cased_L-12_H-768_A-12/1" target="_blank">bert_cased_L-12_H-768_A-12</a>這個版本<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-ZwXOdgM8S_k/XnSTV3Oi1iI/AAAAAAAAVB4/LXinbbj92acCdDZxCAIxMY3weKq402H_ACLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-20%2Bat%2B5.53.23%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="793" data-original-width="1600" height="316" src="https://1.bp.blogspot.com/-ZwXOdgM8S_k/XnSTV3Oi1iI/AAAAAAAAVB4/LXinbbj92acCdDZxCAIxMY3weKq402H_ACLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-20%2Bat%2B5.53.23%2BPM.png" width="640" /></a></div>
<br />
<br />
接著可以看到幾個功能<br />
1.下載模型離線使用(Download Model)<br />
2.打開Colab觀看範例(Open Colab Notebook)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-PYjm4d8P5t8/XnSwZboHqII/AAAAAAAAVCE/xRauI2J9j-kKMHQ6zet53cdKbw_D-yjvwCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-20%2Bat%2B7.59.58%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="993" data-original-width="1600" height="396" src="https://1.bp.blogspot.com/-PYjm4d8P5t8/XnSwZboHqII/AAAAAAAAVCE/xRauI2J9j-kKMHQ6zet53cdKbw_D-yjvwCLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-20%2Bat%2B7.59.58%2BPM.png" width="640" /></a></div>
<br />
<br />
如上圖,如果我們選擇2.直接看範例,則可以執行Run all(如上圖範例),就可以進一步了解模型運作結果,以這個範例為例是以BERT去分類電影評論的情緒分析。<br />
<br />
<br />
接著再往下看像是<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/--3FVAJrH2pk/XnSw_UmIRDI/AAAAAAAAVCM/LFMfe9gpPJQVj-O9CXhZzS6ziVJa0lZqgCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-20%2Bat%2B8.02.29%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="923" data-original-width="1600" height="368" src="https://1.bp.blogspot.com/--3FVAJrH2pk/XnSw_UmIRDI/AAAAAAAAVCM/LFMfe9gpPJQVj-O9CXhZzS6ziVJa0lZqgCLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-20%2Bat%2B8.02.29%2BPM.png" width="640" /></a></div>
<br />
<br />
透過Overview了解模型功能<br />
透過Example use直接使用模型<br />
透過Inputs、Outputs了解輸入與輸出<br />
<br />
特別注意的是每一個模型都會有輸入、輸出的樣態<br />
所以資料在設計上要注意輸入的類型或格式<br />
<br />
這樣就可以直接享用模型囉,在未來不一定是資料科學家<br />
只要了解概念與用途,就可以直接使用模型。<br />
<br />
<br />
最後就是Google TensorFlow Hub也支持上架模型,只要四個步驟<br />
1.建立模型<br />
2.打包模型<br />
3.撰寫文件<br />
4.上架<br />
詳細請參閱這裡<br />
<a href="https://github.com/tensorflow/hub/tree/master/tfhub_dev">https://github.com/tensorflow/hub/tree/master/tfhub_dev</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-GEEsBQ7TQCU/XnSxNAcT90I/AAAAAAAAVCQ/K0LlUU-oKl8n6CIwpXC3UrN5DWViOP2zQCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-20%2Bat%2B8.03.40%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="780" data-original-width="1600" height="156" src="https://1.bp.blogspot.com/-GEEsBQ7TQCU/XnSxNAcT90I/AAAAAAAAVCQ/K0LlUU-oKl8n6CIwpXC3UrN5DWViOP2zQCLcBGAsYHQ/s320/Screen%2BShot%2B2020-03-20%2Bat%2B8.03.40%2BPM.png" width="320" /></a></div>
<br />
<br />
<br />
未來的趨勢就是,大家可以貢獻模型,提升全球在資料科學上的模型交流,筆者覺得是一個很棒的事情。<br />
<br />
<br />
2020年TensorFlow相關文章:<br />
<a href="https://dataology.blogspot.com/2020/03/2020tensorflow-tfhub.html" rel="nofollow" target="_blank">五分鐘掌握2020的TensorFlow新發展-TF_Hub模型篇</a><br />
<a href="https://dataology.blogspot.com/2020/03/2020tensorflow.html" rel="nofollow" target="_blank">五分鐘掌握2020的TensorFlow新發展-生態圈篇</a>資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-48985402761986224602020-03-12T22:47:00.001+08:002020-03-17T10:57:12.869+08:00五分鐘掌握2020的TensorFlow新發展-生態圈篇很快的2020的<a href="https://youtu.be/HlBGYxO8RaU" target="_blank">TensorFlow研討會</a>又開始了,筆者先整理目前至今最新的<br />
TensorFlow生態系統,跟相關連結給大家先瞧瞧,後續再整理這次研討會的發現。<br />
<a href="https://1.bp.blogspot.com/-1sF-MrkQ-gw/XmpIavp2cgI/AAAAAAAAVBY/cyn-z2D4BlMzzVC2wnigD38QMnZp31g7wCLcBGAsYHQ/s1600/01.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="736" data-original-width="1308" height="360" src="https://1.bp.blogspot.com/-1sF-MrkQ-gw/XmpIavp2cgI/AAAAAAAAVBY/cyn-z2D4BlMzzVC2wnigD38QMnZp31g7wCLcBGAsYHQ/s640/01.png" width="640" /></a><br />
<br />
<a href="https://www.tensorflow.org/overview" target="_blank">TensorFlow Core</a>:TF核心程式碼<br />
<a href="https://www.tensorflow.org/js" target="_blank">TensorFlow JS</a>:可應用JS<br />
<a href="https://www.tensorflow.org/lite" target="_blank">TensorFlow Lite</a>:可應用手機、邊緣計算<br />
<a href="https://www.tensorflow.org/lite/microcontrollers" target="_blank">TensorFlow Lite Micro</a>:可應用單晶片或更小的處理器<br />
<a href="https://www.tensorflow.org/tensorboard" target="_blank">TensorFlow Board</a>:視覺化工具<br />
<a href="https://www.tensorflow.org/hub" target="_blank">TensorFlow Hub</a> 可重複利用的模型平台<br />
<a href="https://www.tensorflow.org/tfx" target="_blank">TensorFlow Extended</a> 管理資料流程<br />
<a href="hhttps://www.tensorflow.org/api_docs/python/tf/keras" target="_blank">tf.keras</a>、<a href="https://www.tensorflow.org/api_docs/python/tf/data/Dataset" target="_blank">tf.data</a>、TF Runtime 能夠幫助我們快速分析的TF套件<br />
<a href="https://colab.research.google.com/" target="_blank">Colab </a>:這個超級夯的線上的Google版 Ipython Notebook<br />
<a href="https://www.tensorflow.org/tfrc" target="_blank">TensorFlow Research Cloud</a> 專門給研究用免費使用TPU的計畫<br />
<a href="https://www.tensorflow.org/mlir" target="_blank">MLIR</a>:高效能的機器學習編譯器<br />
<a href="https://www.tensorflow.org/lattice" target="_blank">TensorFlow Lattice</a>:可以用來解釋模型的工具<br />
<a href="https://www.tensorflow.org/model_optimization" target="_blank">TensorFlow Optimization Toolkit </a>:應用在邊緣計算設備的模型的效能優化工具<br />
<a href="https://www.tensorflow.org/graphics" target="_blank">TensorFlow Graphics</a>:可以被用來各種不同的神經網路的處理函式,像是convolutions<br />
<a href="https://www.tensorflow.org/probability" target="_blank">TensorFlow Probability (TFP)</a> :可以直接將機率模型與深度學習整合到TPU、GPU<br />
<a href="https://github.com/tensorflow/tensor2tensor" target="_blank">Tensor2Tensor</a>:應用於Google Brain團隊加速ML研究的套件<br />
<a href="https://github.com/tensorflow/agents" target="_blank">TensorFlow Agents </a>:應用於強化學習的套件<br />
<a href="https://github.com/google/dopamine" target="_blank">Dopamine</a>:新的學習框架應用於強化學習<br />
<a href="https://github.com/deepmind/trfl" target="_blank">TRFL</a>: 可以在GPU上運作的強化學習套件<br />
<a href="https://github.com/tensorflow/mesh" target="_blank">Mesh-TensorFlow</a>:平行處理能力的套件<br />
<a href="https://www.tensorflow.org/guide/ragged_tensor" target="_blank">Ragged tensors</a>:能夠處理不同資料的張量<br />
<a href="https://github.com/tensorflow/ranking" target="_blank">TensorFlow (Learning-to-Rank)</a>:專門用做模型排序<br />
<a href="https://magenta.tensorflow.org/" target="_blank">Magenta</a>:專門用於音樂相關的機器學習套件<br />
<a href="https://github.com/google/nucleus" target="_blank">Nucleus</a>:專門用於DNA分析的套件<br />
<a href="https://www.tensorflow.org/federated" target="_blank">TensorFlow Federated</a>:用於聯合學習(Federated Learning)的套件<br />
<a href="https://github.com/tensorflow/privacy" target="_blank">TensorFlow Privacy</a>:針對Privacy機器學習相關的套件<br />
<a href="https://github.com/tensorflow/fairness-indicators" target="_blank">TensorFlow fairness</a>:針對機器學習公平性指標的分析套件<br />
<a href="https://github.com/deepmind/sonnet" target="_blank">Sonnet </a>:用於建構複雜神經網路的套件<br />
<a href="https://www.tensorflow.org/neural_structured_learning" target="_blank">Neural Structured Learning (NSL) </a>:一種神經結構化學習方法的套件<br />
<a href="https://github.com/google/jax" target="_blank">JAX</a>:專門用於GPU、TPU高效能運算的套件<br />
<a href="https://www.tensorflow.org/quantum" target="_blank">TensorFlow Quantum</a>:用於量子模型的機器學習框架<br />
<a href="https://community.home-assistant.io/t/tensorflow-on-hassio/95255" target="_blank">Hass.io Add-on</a>:用於圖像處理的套件<br />
<br />
<br />
筆者: <a href="https://developers.google.com/community/experts/directory/profile/profile-jerry_wu" target="_blank">JerryWu - Google Developers Experts (Machine Learning)</a> 、APMIC OpenTalk創辦人兼執行長資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-31659298352130176462020-03-06T20:36:00.000+08:002020-03-06T22:22:32.703+08:00五分鐘學會在Colab上使用免費的TPU訓練模型哈囉大家好,雖然忙碌,還是趁空擋想跟大家分享關於<a href="https://github.com/tensorflow/tensorflow/releases/tag/v2.1.0">TensorFlow2.1</a>.x系列的兩三事,一般來說做機器學習模型最需要的就是運算資源,而除了GPU之外,大家一定很想使用Google所推出的<a href="https://cloud.google.com/tpu?hl=zh-tw">Google Cloud TPU</a>來做機器學習模型,重點它很貴,能不能免費的使用他呢?使用<a href="https://colab.research.google.com/">Colab</a>就是首選了。<br />
<div>
<br /></div>
<div>
而這次TF2.1新版推出之後,TPU使用有了一個很大幅度的改版,在<a href="https://colab.research.google.com/" target="_blank">Colab</a>上使用<a href="https://cloud.google.com/tpu?hl=zh-tw" target="_blank">TPU</a>已經變得非常的容易與簡單,這邊就讓Jerry老師來跟大家娓娓道來。<br />
<div>
<br /></div>
<div>
首先不要忘記在Edit -> Notebook settings 要把硬體調整成TPU,如下圖:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-v2-vQ4vUo8w/XmI43r3VkJI/AAAAAAAAVAY/dZqitM2lCe4Hq1BGygbvpUw54P8TRzPDQCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-06%2Bat%2B7.48.59%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="612" data-original-width="882" height="222" src="https://1.bp.blogspot.com/-v2-vQ4vUo8w/XmI43r3VkJI/AAAAAAAAVAY/dZqitM2lCe4Hq1BGygbvpUw54P8TRzPDQCLcBGAsYHQ/s320/Screen%2BShot%2B2020-03-06%2Bat%2B7.48.59%2BPM.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
然後就會獲得TPU資源,如下圖:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-e5GiSNDL9wk/XmI5SPMQ6FI/AAAAAAAAVAg/G14LQERIZTUzXnIK05YIw6Rm2jikA2W1QCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-06%2Bat%2B7.50.52%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="588" data-original-width="800" height="235" src="https://1.bp.blogspot.com/-e5GiSNDL9wk/XmI5SPMQ6FI/AAAAAAAAVAg/G14LQERIZTUzXnIK05YIw6Rm2jikA2W1QCLcBGAsYHQ/s320/Screen%2BShot%2B2020-03-06%2Bat%2B7.50.52%2BPM.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div>
<br />
<div>
<br /></div>
<div>
首先幾個在Colab上要跑TPU的API要了解一下:</div>
<div>
1.<span style="background-color: #fffffe; font-family: monospace , "menlo" , "monaco" , "courier new" , monospace; font-size: 14px; white-space: pre;">tf.distribute.cluster_resolver.TPUClusterResolver()</span></div>
<div>
<span style="font-family: monospace , "menlo" , "monaco" , "courier new" , monospace;"><span style="background-color: #fffffe; font-size: 14px; white-space: pre;">這是負責偵測TPU,可以透過該指令了解Google TPU集群的狀況。</span></span></div>
<div>
<span style="font-family: monospace , "menlo" , "monaco" , "courier new" , monospace;"><span style="background-color: #fffffe; font-size: 14px; white-space: pre;"><br /></span></span></div>
<div>
<span style="font-family: monospace , "menlo" , "monaco" , "courier new" , monospace;"><span style="background-color: #fffffe; font-size: 14px; white-space: pre;">2.</span></span>tf.config.experimental_connect_to_cluster(tpu)</div>
<div>
這是負責連結Google雲端的TPU環境</div>
<div>
<br /></div>
<div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
3.tf.tpu.experimental.initialize_tpu_system(tpu)</div>
</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
這是負責初始化TPU的裝置</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
所以前置作業如下:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-CRj4baTn25k/XmJANeQHHlI/AAAAAAAAVAs/oVfPvzXhBz0q9iF0WOKmYgpzinDQbzk9gCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-06%2Bat%2B8.20.55%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="254" data-original-width="1600" height="99" src="https://1.bp.blogspot.com/-CRj4baTn25k/XmJANeQHHlI/AAAAAAAAVAs/oVfPvzXhBz0q9iF0WOKmYgpzinDQbzk9gCLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-06%2Bat%2B8.20.55%2BPM.png" width="640" /></a></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
必須設定TensorFlow2.x 以及設定tpu的變數。</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
接下來要使用TPU,目前模型有固定的寫法,必須寫成函式,舉例如下:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-wpN5ZnPR1W0/XmJAoYjwrJI/AAAAAAAAVA0/ZgQDclqOmzYuts9-NmT6C7-JcQ67HYWfQCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-06%2Bat%2B8.22.45%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="284" data-original-width="1474" height="122" src="https://1.bp.blogspot.com/-wpN5ZnPR1W0/XmJAoYjwrJI/AAAAAAAAVA0/ZgQDclqOmzYuts9-NmT6C7-JcQ67HYWfQCLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-06%2Bat%2B8.22.45%2BPM.png" width="640" /></a></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
資料的部分也是要寫成函式,這邊範例是採用mnist,所以舉例如下:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-4rJpHDFKi-A/XmJA3pxfx_I/AAAAAAAAVA4/RQ6vOs3SYnQQGGatD-U-_5ujI-NRbDIPgCLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-06%2Bat%2B8.23.47%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="546" data-original-width="1426" height="244" src="https://1.bp.blogspot.com/-4rJpHDFKi-A/XmJA3pxfx_I/AAAAAAAAVA4/RQ6vOs3SYnQQGGatD-U-_5ujI-NRbDIPgCLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-06%2Bat%2B8.23.47%2BPM.png" width="640" /></a></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
最後是訓練模型的程式碼,有個重點是</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<div style="line-height: 19px;">
4.tf.distribute.experimental.TPUStrategy</div>
</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
是呼叫前面TPU分配策略的指令,所以要搭配在程式碼當中</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
5.strategy.scope()</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
搭配這段程式碼,可以把模型與要計算的優化器、衡量標準放到運算引擎上</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
如下:</div>
</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-ESrdwumvSss/XmJBC--m3FI/AAAAAAAAVA8/kRe1R-wSc0E-fj7ZBDwcNnXXMmZ68KoYACLcBGAsYHQ/s1600/Screen%2BShot%2B2020-03-06%2Bat%2B8.24.23%2BPM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="534" data-original-width="1540" height="220" src="https://1.bp.blogspot.com/-ESrdwumvSss/XmJBC--m3FI/AAAAAAAAVA8/kRe1R-wSc0E-fj7ZBDwcNnXXMmZ68KoYACLcBGAsYHQ/s640/Screen%2BShot%2B2020-03-06%2Bat%2B8.24.23%2BPM.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
目前實測的結果,如果以<span style="background-color: #fffffe; font-family: monospace , "menlo" , "monaco" , "courier new" , monospace; font-size: 14px; white-space: pre;">mnist資料為例,</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="background-color: #fffffe; font-family: monospace , "menlo" , "monaco" , "courier new" , monospace; font-size: 14px; white-space: pre;">TPU:36秒</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="background-color: #fffffe; font-family: monospace , "menlo" , "monaco" , "courier new" , monospace; font-size: 14px; white-space: pre;">GPU(P100):90秒</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: monospace , "menlo" , "monaco" , "courier new" , monospace;"><span style="background-color: #fffffe; font-size: 14px; white-space: pre;">CPU:291秒</span></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: monospace , "menlo" , "monaco" , "courier new" , monospace;"><span style="background-color: #fffffe; font-size: 14px; white-space: pre;"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: monospace , "menlo" , "monaco" , "courier new" , monospace;"><span style="background-color: #fffffe; font-size: 14px; white-space: pre;">使用心得是,程式碼不用改動太多,在TensorFlow 2.x的架構下,可以很快的進行訓練,程式碼也可以很快速地調整,</span></span><span style="background-color: #fffffe; font-family: monospace , "menlo" , "monaco" , "courier new" , monospace; font-size: 14px; white-space: pre;">以後大家就可以用前所未有的速度訓練模型囉!</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="background-color: #fffffe; font-family: monospace , "menlo" , "monaco" , "courier new" , monospace; font-size: 14px; white-space: pre;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: monospace , "menlo" , "monaco" , "courier new" , monospace;"><span style="background-color: #fffffe; font-size: 14px; white-space: pre;">當然如果您很想更近一步了解</span></span>Google Cloud Platform上面的TPU費用,可以參考下面連結</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://cloud.google.com/tpu/docs/types-zones">https://cloud.google.com/tpu/docs/types-zones</a></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://cloud.google.com/tpu/pricing">https://cloud.google.com/tpu/pricing</a></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<br /></div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
相關程式碼請參考</div>
<div style="background-color: #fffffe; font-family: monospace, Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;">
<a href="https://github.com/jerrywu2013/TensorFlow_TPU" style="white-space: normal;">https://github.com/jerrywu2013/TensorFlow_TPU</a><br />
<br /></div>
</div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com1tag:blogger.com,1999:blog-5898372732399324805.post-91892331955122247012020-02-06T09:15:00.001+08:002020-03-07T23:40:35.954+08:00關於Jerry老師的強化學習課程最近Jerry老師即將準備完成的新課程,關於強化學習,歡迎大家一起來學習<br />
優惠名額有限動作要快:)<br />
請點<a href="https://hiskio.com/courses/325?promo_code=6G6N26G" target="_blank">預購連結</a><br />
<br />
<br />
<br />資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-77927691582642797622019-09-09T00:26:00.001+08:002019-09-09T08:57:46.402+08:00[ML Study Jam] How Google does Machine Learning的Module 3 Quiz學習日誌之企業導入機器學習的方法How Google does Machine Learning的學習日誌<br />
<br />
最近Google連續辦了兩次ML Study Jam的活動,包含<br />
<a href="https://events.withgoogle.com/ml-study-jam-basic-tw/" target="_blank">ML Study Jam 機器學習培訓計劃- 初級- 首頁</a>,以及<a href="https://events.withgoogle.com/ml-study-jam-interm-tw/" target="_blank">ML Study Jam 機器學習培訓計劃- 進階-首頁</a>,這兩階段的課程內容,個人覺得非常適合企業在導入ML給同仁必上的內容,尤其適合非科班出身的又想轉型的朋友們。<br />
<br />
在進階課程的當中,筆者覺得在「<a href="https://www.coursera.org/learn/google-machine-learning" target="_blank">How Google does Machine Learning</a>」當中,有許多有趣的題目與內容,是自己多年做資料科學服務有所共鳴的,希望能夠透過部落格分享學習的心得,提供給想導入ML的企業與想持續學習導入ML的朋友。(請注意本篇文章僅針對部分題目提供心得分享,並非答案分享)<br />
<br />
<br />
過去筆者在導入多年的資料科學服務當中,發現資料科學家要有一個非常重要的能力,是能夠快速辨識這家企業與題目,到哪一個階段,接著再給予對應的服務內容,避免導入過程資源的浪費,而課程當中Google整理了一個「Path to ML」來幫忙大家釐清企業的ML導入階段。<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-LduJgcQIBk8/XXUqfkI6W_I/AAAAAAAAU7M/uNqyVdOiDiEZNhEjAWqQP-M0EAJDctKvwCLcBGAs/s1600/Screen%2BShot%2B2019-09-09%2Bat%2B12.20.57%2BAM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="733" data-original-width="1600" height="291" src="https://1.bp.blogspot.com/-LduJgcQIBk8/XXUqfkI6W_I/AAAAAAAAU7M/uNqyVdOiDiEZNhEjAWqQP-M0EAJDctKvwCLcBGAs/s640/Screen%2BShot%2B2019-09-09%2Bat%2B12.20.57%2BAM.png" width="640" /></a></div>
<br />
在Module 3 Quiz的部分,有一題要學員能夠針對題目,以「Path to ML」五階段做辨認,而其中五階段包含「Individual contributor」、「Delegation」、「Digitization」、「Big data and Analytics」、「Machine learning」,在ML and Business Processes課程當中,個人覺得非常好的一個觀念是,企業要導入一個從無ML->有ML的過程,必須先找到流程上的輸入與輸出,接著在輸出中透過數據與ML找出具有洞見的結果,將結果回到流程上去改善輸出。<br />
<br />
因此回到「Path to ML」,我們首先要能夠辨認,我們在企業流程中,到底是哪一個階段,才能去藉由階段工作來正確導入ML,筆者整理如下:<br />
<br />
I for individual contributor:任務的本體,如單一櫃員<br />
D for delegation:多個可執行任務的人,如多個店員<br />
Z for digitization:數位化的工具,如ATM,可以透過機器進行重複化的工作<br />
B for big data and analytics:透過資料挖掘洞見,提升營運效率,如汽車生產<br />
M for machine learning:透過前面的數據,以機器學習來自動化改善整個營運效益,如推薦系統<br />
<br />
聰明的朋友,如果題目如下,您會將上述的I,D,Z,B,M,分別放在哪裡呢?<br />
<br />
問題1<br />
製造工廠有許多機械手臂,整個生產團隊透過手臂上收集的數據來幫助確定生產的時間表。<br />
問題2<br />
一個客戶服務團隊持續接收在線上的訂單<br />
問題3<br />
一個圖書館員在城市圖書館工作<br />
問題4<br />
停車收費系統根據可能預期的需求來自動改變費率<br />
問題5<br />
自動電子收費器透過讀取汽車牌照跟駕駛索取過路費(如ETC)<br />
<br />
問題1看起來非常明顯是以數據來挖掘洞見,問題2與問題3很容易理解是傳統的運作階段,問題4與問題5比較容易會有誤會是在「digitization」與「machine learning」的差別,但仔細一看,事實上ML有個關鍵是透過數據「自動化」原先的流程,因此會獲得一個不斷改變的輸出,答案就出來了。<br />
<br />
<br />
看起來大家都有十足的把握了,我們繼續看一下去,如果題目如下,您們的選擇為何?<br />
<br />
題目6<br />
所有任務,包含執行參數、選擇參數、參數回饋都是自動化<br />
題目7<br />
一個人在執行任務,而這個任務與指令來自人與人之間的方式運作<br />
題目8<br />
一個電腦執行任務,而軟體工程師透過歷史的模式,來進行參數的調整。<br />
題目9<br />
一個電腦執行過程是透個人從介面上輸入參數來進行。<br />
題目10<br />
一個任務是從一群人來完成<br />
<br />
問題6我們可以發現全數自動化,會是機器學習的強項,答案是非常明確的。接著在問題7與問題10的部分,也很容易辨識是傳統1個與多個人的運作方式,最後這次的題目8與題目9分別就相對明顯很多,題目9單純只有透過介面來做互動,而題目8則透過數據挖掘模式來進行參數調整。<br />
<br />
<br />
最後在Quiz最尾端的題目也非常有趣,題目大意是在說一家企業需要在一年內導入ML,我們是這家企業的顧問,而我們應該請該企業集中資源在哪一個地方,譬如說:「定義關鍵目標」、「收集資料」、「建立基礎設施」、「最佳化演算法」、「整合模型」。<br />
<br />
其實有經驗的顧問會選擇「收集資料」,因為這是所有企業導入ML的最重要的問題,尤其以筆者過去的經驗是,傳統企業常常是因為沒有資料而無法導入ML。而其他都不重要嗎?並不是,而是沒有資料,對於定好目標、建立設施、最佳化算法、整合模型就都沒有了意義。<br />
<br />
但是筆者提一個觀點是,如果我們企業要導入ML,也不能盲目的收集資料,而是要了解解決什麼問題。<br />
所以回過頭來,一樣可以觀察公司目前的階段,從沒有ML->有ML,我們在「Individual contributor」、「Delegation」、「Digitization」、「Big data and Analytics」、「Machine learning」是哪一個階段,才能將資源妥善運用在導入ML的正確方向當中。<br />
<br />
以上是筆者對於「How Google does Machine Learning」的Module 3 Quiz拙見,歡迎討論。<br />
<div>
<br /></div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0Taipei, Taiwan25.0329694 121.5654177000000124.5725269 120.91997070000001 25.493411899999998 122.21086470000002tag:blogger.com,1999:blog-5898372732399324805.post-72461195380095709542018-01-26T11:36:00.002+08:002019-09-09T09:13:46.306+08:00歡迎加入機器智能實驗室_FB社團大家好,我是資料科學實驗室作者Jerry,由於個人工作歷程的發展,目前正在從「數據科學」轉移到「機器智能」上發展,所以建立了「<a href="https://www.facebook.com/groups/miailab/" target="_blank">機器智能實驗室</a>」,在這裡不同以往的,我們可以有更多的互動與交流,也歡迎大家一起進入我們的研究行列。<br />
<br />
未來所有智能化的來源都是資料,而研究多年「資料應用」的Jerry,希望能夠將資料與智能化整合,就像阿里巴巴<a href="http://www.sohu.com/a/145209460_468734" target="_blank">馬雲</a>說的「<span style="color: #212121; font-family: inherit; font-size: 16px; white-space: pre-wrap;">以前的電器是插上了電以後就聽你話。未來電器不僅要通電,更要通數據。機器收到數據之後,機器比人變得越來越聰明」。</span><br />
<br />
<span style="color: #212121; font-family: inherit; font-size: 16px; white-space: pre-wrap;">另外也有牛津大學的學者提到「<a href="http://xn--4gq9qu6ava822fywijmgmrfywkijal2a32c8a950imy9b7d9agc2f/" target="_blank">機器智能將會是我們人類最後一個發明</a>」,讓我更決定開始投入</span>機器智能(Machine Intelligence)的相關研究。<br />
<br />
在接下來幾年,資料將會是企業最重要的資產,為了能夠掌握未來發展,企業應該要能夠掌握資料,並開始投入研究資料,進而將資料用在各種智能應用上面。但是因為機器智能(Machine Intelligence)技術越來越複雜,門檻越來越高,因此希望成立<a href="https://www.facebook.com/groups/miailab/" target="_blank">機器智能實驗室</a>,希望大家一同共襄盛舉。<br />
<br />
<br />
<br />
<br />
======================= [工商服務] =======================<br />
目前JerryWu老師是<a href="https://google-developers.appspot.com/community/experts/directory/profile/profile-jerry_wu" target="_blank">Google機器學習開發專家</a>,同時擔任機器智能(人工智慧)訓練講師,目前在廣宣學堂、T客邦、資策會、工研院、中國生產力中心、文化大學進修推廣部......等知名訓練機構擔任講師,也同時是<a href="https://www.ap-mic.com/zh-hant/" target="_blank">亞太智能機器</a>的創辦人兼技術長。<br />
教授課程包含:<br />
<a href="https://broadmission.kktix.cc/events/446743e4-abfad9-6a6f5a-f42d10-714081-622421-8159dc-2fdf73-ce3581" target="_blank">機器學習</a>、<a href="https://broadmission.kktix.cc/events/cfb4fc66" target="_blank">機器理解</a>、<a href="https://www.techbang.com/posts/56748-course-machine-vision-principle-and-implementation-from-image-processing-to-dynamic-analysis-object-tracking-using-python-opencv-to-create-practical-applications" target="_blank">機器視覺</a>、Python基礎課程、Python資料分析課程、預測分析與管理。<br />
<br />
有需要的朋友歡迎來信 jerry@ap-mic.com<br />
<br />
<br />
<br />資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-56116884681666597302017-05-08T14:20:00.000+08:002017-05-08T18:53:42.066+08:00透過雲端工具快速實作情緒分析(Sentiment Analysis)最近實在是太忙,終於有一點時間可以發文,情緒分析一直是筆者的研究之一。自從兩年前「<a href="http://dataology.blogspot.tw/2015/04/sentiment-analysis.html" target="_blank">情緒分析(Sentiment Analysis)的作法與商業價值</a>」獲得高度迴響之後,就一直要找時間分享相關的研究。而剛好這兩年Google推出的自然語言技術越來越成熟,筆者就把使用心得給記錄在這邊。<br />
<br />
因此本次實作主要使用<a href="https://cloud.google.com/natural-language/" target="_blank">Google自然語言API (Cloud Natural Language API)</a>來進行情緒辨識,目前支援簡體中文、繁體中文、英文、法文、德文、義大利文、日文、韓文、葡萄牙文、西班牙文。<br />
<br />
<br />
<div class="separator" style="clear: both;">
如果您只是是玩玩,想看一下Demo,Google提供兩個地方給大家玩,第一個到<a href="https://cloud.google.com/natural-language/" target="_blank">官網</a>。</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-zzARUUqRfto/WRAIx4PnBtI/AAAAAAAATkk/k52cVHOPkDcxAL3FgkleBpAGpzdSbNF9gCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.56.33.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="338" src="https://3.bp.blogspot.com/-zzARUUqRfto/WRAIx4PnBtI/AAAAAAAATkk/k52cVHOPkDcxAL3FgkleBpAGpzdSbNF9gCLcB/s640/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.56.33.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
可以自行輸入文字進行辨識情緒。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
第二個</div>
<div class="separator" style="clear: both; text-align: left;">
或是進到Google API進行API測試,網址請點<a href="https://developers.google.com/apis-explorer/?hl=zh_TW#p/language/v1/language.documents.analyzeSentiment" target="_blank">這裡</a><a href="https://2.bp.blogspot.com/-S-qE0uTnFUI/WRAJnSO3mII/AAAAAAAATks/0TWpTSaCzdcW61DcWpsBt0gTV271yU0tgCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B14.00.36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="174" src="https://2.bp.blogspot.com/-S-qE0uTnFUI/WRAJnSO3mII/AAAAAAAATks/0TWpTSaCzdcW61DcWpsBt0gTV271yU0tgCLcB/s640/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B14.00.36.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
透過上方的輸入之後,可以獲得情緒分數的回傳</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-1XEhLs9o688/WRAJ0jhNELI/AAAAAAAATkw/xBU4fenGmZMF5OV0cpMy8UrCFjYWe5ZIgCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B14.01.43.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="295" src="https://3.bp.blogspot.com/-1XEhLs9o688/WRAJ0jhNELI/AAAAAAAATkw/xBU4fenGmZMF5OV0cpMy8UrCFjYWe5ZIgCLcB/s320/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B14.01.43.png" width="320" /></a></div>
<div class="separator" style="clear: both;">
<br /></div>
<br />
第三個就是今天要說的,透過雲端工具來取得情緒分數,做之後的系統運用。<br />
<br />
在基本教學前,我們自己的電腦必須準備的環境如下:<br />
<br />
<ol>
<li>Google Cloud Platform帳號</li>
<li>Python 3以上的版本</li>
<li>Virtualenv (pip install virtualenv)</li>
<li><a href="https://cloud.google.com/sdk/docs/quickstart-mac-os-x" target="_blank">gcloud(</a>依據自己的電腦做安裝,本次範例採用Mac OS X)</li>
<li>務必完成<a href="https://cloud.google.com/sdk/docs/quickstart-mac-os-x" target="_blank">gcloud上面的登入設定</a></li>
</ol>
<div>
首先建立一個虛擬環境,筆者用gNLP為名稱,接著進入虛擬環境,指令如下圖所示:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-wIgYz19_KUw/WRAAAPo0g5I/AAAAAAAATjk/v5yNPLrVKkkZVEZ9E47Ng1YR6kHNPbrZgCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.19.31.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="140" src="https://2.bp.blogspot.com/-wIgYz19_KUw/WRAAAPo0g5I/AAAAAAAATjk/v5yNPLrVKkkZVEZ9E47Ng1YR6kHNPbrZgCLcB/s640/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.19.31.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們要裝一下gcloud的Python套件,我們可以只裝Language的相關套件就行。</div>
<div class="separator" style="clear: both; text-align: left;">
pip3 install --upgrade google-cloud-language</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-OlUhIt7P8gY/WRACBAMLGrI/AAAAAAAATjw/4EGBfjL55x4Dj-pOnoh6NwrTIswF9o6kACLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.28.12.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="236" src="https://1.bp.blogspot.com/-OlUhIt7P8gY/WRACBAMLGrI/AAAAAAAATjw/4EGBfjL55x4Dj-pOnoh6NwrTIswF9o6kACLcB/s640/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.28.12.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著測試一下google cloud的套件是否正常,進入Python之後打以下指令:</div>
<div class="separator" style="clear: both; text-align: left;">
from google.cloud import language</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-gEQmqWvQ4yM/WRACWq_eI4I/AAAAAAAATj0/WsE4CAKIqSUgyEb5Y8BTwkcjUY1ydv2ZwCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.29.33.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="94" src="https://3.bp.blogspot.com/-gEQmqWvQ4yM/WRACWq_eI4I/AAAAAAAATj0/WsE4CAKIqSUgyEb5Y8BTwkcjUY1ydv2ZwCLcB/s640/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.29.33.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著可以下載筆者從 範例加工過的程式碼,請點<a href="https://github.com/jerrywu2013/GoogleComputeEngine/blob/master/GCP_Cloud%20Natural%20Language%20API/gnlp.py" target="_blank">網址</a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-qlOoq28RhcU/WRADDlNpWyI/AAAAAAAATj8/UsXL5NXRb3EgdmE0UbNrWUrizq85JxlRQCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.32.48.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="340" src="https://2.bp.blogspot.com/-qlOoq28RhcU/WRADDlNpWyI/AAAAAAAATj8/UsXL5NXRb3EgdmE0UbNrWUrizq85JxlRQCLcB/s640/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.32.48.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
這時候直接執行英文句子沒問題,但是輸入中文句子,就會出現問題,像是下面這張圖所示</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-u90_8mWaZ6Y/WRAEQnzNZ6I/AAAAAAAATkE/k9waRC1IJM4nvyCQr3P0KtvehhWH6SgwwCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.37.35.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="156" src="https://4.bp.blogspot.com/-u90_8mWaZ6Y/WRAEQnzNZ6I/AAAAAAAATkE/k9waRC1IJM4nvyCQr3P0KtvehhWH6SgwwCLcB/s640/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.37.35.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
大意是說他抓不到回傳的資訊,不過事實是目前中文正在測試,他不回傳您資料,但是有其他撇步可以使用。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們要到套件本身做一下修改,從上面我們可以看到他是呼叫document這個函式,所以我們要到"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/language/document.py" 去修改 「document.py」這一個檔案。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
打開之後 搜尋「null」看到下面的畫面</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-P0MJO03eOG0/WRAE7Km1BXI/AAAAAAAATkM/9-DCTyi_LcERGF7ks2mFY8eMItvYciSAgCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.40.03.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="144" src="https://1.bp.blogspot.com/-P0MJO03eOG0/WRAE7Km1BXI/AAAAAAAATkM/9-DCTyi_LcERGF7ks2mFY8eMItvYciSAgCLcB/s640/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.40.03.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
將裡面的null改成en,讓google自動偵測語言的機制有了預設值,改完之後存檔案。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
再回來執行程式,就大功告成。</div>
<div class="separator" style="clear: both; text-align: left;">
範例如下</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-JxJrDcsxaj8/WRAF8K9GuRI/AAAAAAAATkY/coaQ-sHF_wcqk5JWmSJpKIg420M3i4d7wCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.45.08.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="83" src="https://1.bp.blogspot.com/-JxJrDcsxaj8/WRAF8K9GuRI/AAAAAAAATkY/coaQ-sHF_wcqk5JWmSJpKIg420M3i4d7wCLcB/s320/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.45.08.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-VsXAWWawG_c/WRAF8KtsYBI/AAAAAAAATkU/RM_8_md35RIxkXsLC8T7MH54pm2GPW1MQCLcB/s1600/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.44.55.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="78" src="https://3.bp.blogspot.com/-VsXAWWawG_c/WRAF8KtsYBI/AAAAAAAATkU/RM_8_md35RIxkXsLC8T7MH54pm2GPW1MQCLcB/s320/%25E8%259E%25A2%25E5%25B9%2595%25E5%25BF%25AB%25E7%2585%25A7%2B2017-05-08%2B13.44.55.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Score是指情緒分數,Magnitude是指情緒的強度。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
記錄一下<a href="https://cloud.google.com/natural-language/" target="_blank">Google自然語言API (Cloud Natural Language API)</a>的API在一定的呼叫次數下是免費的,計費方式是採用每月1,000個字來做單位,以情緒分析為例,大概超過5,000個字之後就是1元美金,但是文字量越多越便宜,實際的計價方式可參考<a href="https://cloud.google.com/natural-language/pricing" target="_blank">官方網址</a>。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
總結一下</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
優點:</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ol>
<li>透過Google CNL API來實作辨識情緒,方便又快速,可以讓大部分的工程師、分析師比較不會太忙(ㄅㄠˋ ㄍ ㄢ )。</li>
<li>不用理基礎設施問題,過往透過辨識情緒要耗費的運算資源,這部分都不用考慮進去,直接以Serverless的概念運作。</li>
<li>可以在五分鐘之內,完成情緒辨識功能,並且在一小時之內上線相關的系統,不用再訓練模型、驗證模型。</li>
</ol>
<div>
缺點:</div>
<div>
<ol>
<li>有些複雜的句子目前還不能有效地辨識,像是筆者輸入「這台車真好,可是我從來都不開」情緒辨識的結果居然是正面的,所以中文句子還是有辨識問題。</li>
<li>一些具有領域知識的名詞 或是特殊名詞 ,混雜在句子裡面的時候,辨識也不會太準,像是「這台車真的是瞎爆了」這樣的句子也無法正常辨識。</li>
<li>透過Google CNL API實作,超過一定的呼叫次數後,就要錢(廢話)。</li>
</ol>
<div>
所以結論還是要請資料科學家幫忙做調整跟修正囉:)</div>
</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div>
<br /></div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-35080262661004696162017-03-05T17:13:00.001+08:002017-03-05T17:18:47.283+08:002017年台灣巨量資料(大數據)、資料科學家相關課程<br />
很快的新的一年又開始了,筆者已經持續整理資料科學相關課程長達三年,今年也一樣整理相關課程與上課時間給有需要的朋友查詢。<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-XXjdqL18jKk/WLvXNxy7c-I/AAAAAAAATbw/JFofeEegKuUsiUOXnhQTvp3filxc70mHQCLcB/s1600/courses-icon-6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="200" src="https://4.bp.blogspot.com/-XXjdqL18jKk/WLvXNxy7c-I/AAAAAAAATbw/JFofeEegKuUsiUOXnhQTvp3filxc70mHQCLcB/s200/courses-icon-6.png" width="200" /></a></div>
<br />
工研院<br />
<ul>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=E13AFD66-5648-45DD-884E-8FF075AC364F" target="_blank">巨量資料資料分析自動化設計-NoSQL(Cassandra)資料庫系列</a>(2017/03/07~2017/03/08)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=F84B448A-BDCB-4353-81BB-7439FEFE4816" target="_blank">巨量資料資料分析自動化設計-R實務應用進階課程</a> (2017/04/13~2017/04/14)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=B0E6A323-3778-48D1-A0C6-7E26AF63E115" target="_blank">巨量資料資料分析自動化設計-製造業專題系列</a> (2017/05/11~2017/05/12)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=36FD08B1-3AD2-432F-B88F-0DC977F77DD2" target="_blank">巨量資料之智慧金融服務設計與分析</a> (2017/05/16~2017/05/17)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=C9635BB3-861C-4601-BC34-18FF5C25FFEF" target="_blank">巨量資料資料分析自動化設計-Python系列</a> (2017/06/05~2017/06/06)</li>
</ul>
<br />
資策會<br />
<ul>
<li><a href="http://www.iiiedu.org.tw/ites/RD.htm" target="_blank">Big Data資料分析二部曲-R軟體與資料探勘</a> (2017/03/18~2017/03/19)</li>
<li><a href="http://www.iiiedu.org.tw/ites/TEXT.htm" target="_blank">文字資料探勘實作班</a> (2017/04/29~2017/04/30)</li>
<li><a href="http://www.iiiedu.org.tw/ites/CBDA.htm" target="_blank">Big Data之處理與分析(Hadoop)進階班</a> (2017/07/15~2017/07/16)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RB.htm" target="_blank">Big Data資料分析首部曲-R軟體實作</a> (2017/05/15~2017/05/16)</li>
<li><a href="http://www.iiiedu.org.tw/ites/PYTHON.htm" target="_blank">Python資料探勘實作</a> (2017/04/15~2017/04/16)</li>
<li><a href="http://www.iiiedu.org.tw/ites/CBD.htm" target="_blank">Big Data之處理與分析(Hadoop)實務班</a> (2017/06/10~2017/06/11)</li>
<li><a href="http://www.iiiedu.org.tw/ites/ELAS.htm" target="_blank">Elasticsearch分散式系統實務班</a> (2017/03/04~2017/03/05)</li>
<li><a href="http://www.iiiedu.org.tw/ites/SPARK.htm" target="_blank">Spark巨量資料分析實務班</a> (2017/04/15~2017/04/16)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RV.htm" target="_blank">Big Data資料分析三部曲-R軟體與資料視覺化</a> (2017/03/25~2017/03/26)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RWP.htm" target="_blank">R軟體與網頁資料擷取應用</a> (2017/06/10~2017/06/11)</li>
<li><a href="http://www.iiiedu.org.tw/ites/TEXT.htm" target="_blank">文字資料探勘實作班</a> (2017/4/29~2017/04/30)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RV.htm" target="_blank">Big Data資料分析三部曲-R軟體與資料視覺化</a> (2017/03/25~2017/03/26)</li>
<li><a href="http://www.iiiedu.org.tw/ites/CBDA.htm" target="_blank">Big Data之處理與分析(Hadoop)進階班</a> (2017/07/15~2017/07/16)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RST.htm" target="_blank">R軟體與串流資料分析</a> (2017/05/20~2017/05/21)</li>
</ul>
<br />
中國生產力中心<br />
<ul>
<li><a href="http://edu.cpc.org.tw/02R/R12.htm" target="_blank">掌握市場脈動數據分析大解密</a> (3/24、5/12、7/14、10/20)</li>
<li><a href="http://edu.cpc.org.tw/02R/R11.htm" target="_blank">預測管理與分析</a> (5/19、7/25、9/15、11/10)</li>
<li><a href="http://edu.cpc.org.tw/02R/R10.htm" target="_blank">大數據於C2B的應用</a> (4/14、6/23、8/25、10/24)</li>
<li><a href="http://edu.cpc.org.tw/02R/R9.htm" target="_blank">物聯網與大數據整合應用下的產業機會</a> (3/17、6/30、9/22、11/24)</li>
</ul>
<br />
三星課程網<br />
<ul>
<li><a href="http://www.tutortristar.com/topic/Data-mining/class-crawler-david.html" target="_blank">網路爬蟲技術Python入門實作</a> (2017/02/18~2017/02/19)</li>
<li><a href="http://www.tutortristar.com/topic/Data-mining/Data-mining-chen.html" target="_blank">大數據資料採礦實作</a> (2017/03/18~2017/03/19)</li>
</ul>
<br />
臺灣資料科學協會<br />
<ul>
<li><a href="http://datasci.tw/event/DM_QuickTour_170114/" target="_blank">資料探勘速遊</a> (2017/01/14)</li>
<li><a href="http://datasci.tw/event/engineer_statistics_170114/" target="_blank">給工程師的統計學及資料分析 123</a> (2017/01/14)</li>
<li><a href="http://datasci.tw/event/step_by_step_r_170114/" target="_blank">手把手教你 R 語言資料分析實務</a> (2017/01/14)</li>
<li><a href="http://datasci.tw/event/step_by_step_dl_170115/" target="_blank">手把手的深度學習實務</a> (2017/01/15)</li>
<li><a href="http://datasci.tw/event/build_trading_business_with_r_170115/" target="_blank">使用 R 語言建立自己的演算法交易事業</a> (2017/01/15)</li>
<li><a href="http://datasci.tw/event/bsp_edu_health_170115/" target="_blank">人類行為大數據分析:資料科學如何應用在教育及醫療領域</a> (2017/01/15)</li>
<li><a href="http://datasci.tw/event/smartcity_stbigdata_170211/?utm_source=TDSF&utm_campaign=smartcity_stbigdata_170211&utm_medium=courses" target="_blank">智慧城市中的時空大數據應用</a> (2017/02/11)</li>
<li><a href="http://datasci.tw/event/manufacturing_170211/?utm_source=TDSF&utm_campaign=manufacturing_170211&utm_medium=courses" target="_blank">製造資料科學:從預測性思維到處方性決策</a> (2017/02/11)</li>
<li><a href="http://datasci.tw/event/ml_170212/?utm_source=TDSF&utm_campaign=ml_170212&utm_medium=courses" target="_blank">Machine Learning 機器學習課程</a> (2017/02/12)</li>
<li><a href="http://datasci.tw/event/step_by_step_dl_170212/?utm_source=TDSF&utm_campaign=step_by_step_dl_170212&utm_medium=courses" target="_blank">手把手的深度學習實務</a> (2017/02/12)</li>
<li><a href="http://datasci.tw/event/statistician_teach_you_r_170211-12/?utm_source=TDSF&utm_campaign=statistician_teach_you_r_170211-12&utm_medium=courses" target="_blank">統計學家教你穩紮穩打 R 語言資料分析</a> (2017/02/11~2017/02/12)</li>
</ul>
<br />
<br />
筆者整理這幾年來有發現一些心得,關於大數據或資料科學家課程,有越來越多元化的趨勢,另外除了資策會與工研院有長期開課之外,有不少新的公司或團體也有類似的課程出現,像是臺灣資料科學協會、中國生產力中心、三星課程網。<br />
<br />
以上 如有遺漏 歡迎留言告知。<br />
<br />
<br />
<br />
<div>
歷年資料參考:</div>
<div>
<a href="http://dataology.blogspot.tw/2016/01/2016.html" target="_blank">2016年台灣巨量資料(大數據)、資料科學家相關課程</a><br />
<a href="http://dataology.blogspot.tw/2015/05/2015-201505.html" target="_blank">2015年台灣巨量資料(大數據)、資料科學家相關課程</a></div>
<div>
<a href="http://dataology.blogspot.tw/2014/01/2014big-data.html" target="_blank">2014年台灣巨量資料、資料科學家相關課程</a></div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-25518586416696200292016-07-17T17:08:00.001+08:002016-07-17T17:16:29.447+08:00「2016 台灣資料科學愛好者年會」精彩資料總整理(持續更新中)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-qu_9Ujv5D9Q/V4tLSMTnMzI/AAAAAAAATE8/VOMzobCEFjs00Uvv5h63uWgnTApvUMD_wCLcB/s1600/11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="259" src="https://1.bp.blogspot.com/-qu_9Ujv5D9Q/V4tLSMTnMzI/AAAAAAAATE8/VOMzobCEFjs00Uvv5h63uWgnTApvUMD_wCLcB/s320/11.png" width="320" /></a></div>
<br />
<br />
這幾天(2016/7/14 ~ 2016/7/17)資料科學界在台灣最重要的活動,莫過於<a href="http://datasci.tw/index.html" target="_blank">2016 台灣資料科學愛好者年會</a>,這已經是每年在台灣資料科學界最大的活動之一,今年的票也是不到一小時就幾乎賣光,對這個活動的歷史有興趣的朋友們可以參考「<a href="http://dataology.blogspot.tw/2015/08/2015.html" target="_blank">2015年</a>」、「<a href="http://dataology.blogspot.tw/2014/08/2014.html" target="_blank">2014年</a>」,今年的主題相較去年,更有趣,更多元、更多產業上的案例。<br />
<br />
除此之外,筆者也發現贊助的公司也變得相當的多,今年贊助公司包含國泰人壽、東森信息科技股份有限公司、玉山金控、funP 雲沛創新集團、HTC、台灣 IBM、新加坡商鈦坦科技 TITANSOFT、潮網科技、Vpon 威朋、宇匯知識、勤業眾信、遊戲橘子、GMobi通用移動、Google、艾肯娛樂、宣捷生物科技、MIGO 功典、Mozilla TW、天氣風險、琅茶,從中我們可以發現不只資訊公司,也包含其他產業的公司,像是生物研究、天氣、會計,甚至是金融業都來加入,整體上台灣資料科學發展的凝聚力,在今年又更上一層樓。<br />
<br />
<br />
今年筆者一樣秉持著自己也是資料科學愛好的一份子,將今年的資料科學活動內容整理如下,供自己與有興趣的朋友們參考:<br />
<br />
<br />
<ol>
<li><a href="http://www.slideshare.net/tw_dsconf/2016-64093833" target="_blank">2016台灣資料科學年會開場投影片</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/riot-games-head-of-data-science" target="_blank">Riot Games Head of Data Science</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64076685" target="_blank">用數據解決都會城市的停車問題</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64076745" target="_blank">隨機對照實驗在公共領域的應用</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64076799" target="_blank">數據分析在運輸管理之應用</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64076883" target="_blank">大數據下的情緒分析</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/2-64076950" target="_blank">一個賭徒的告白 2:交易策略建構與分析,為何你該賭小一點?</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ceo-64077007" target="_blank">從薪酬制度讀 CEO 的行為心理學 </a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64077120" target="_blank">自己的空氣品質自己量 : 談參與式環境感測的機會與挑戰</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64078273" target="_blank">如何備料:資料的抓取、清理以及串接</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64078333" target="_blank">黃從仁/心理與行為資料中的因與果</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64078404" target="_blank">資料科學與防疫應用的結合 : 以登革熱防治為例</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64078610" target="_blank">資料分析前的奏曲 : 談資料收集的挑戰</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/polling-linhow-to-approach-data-science-problems-from-start-to-end" target="_blank">How to approach data science problems from start to end</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/mobility-on-demand-for-future-cities" target="_blank">未來城市的任意門 (Mobility on Demand for Future Cities)</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64090916" target="_blank">人類行為訊號處理 : 跨學科 (醫療、教育、心理) 應用實例分享、心得、展望</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64090957" target="_blank">從手機解讀行為與心理</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/crowd-computing-for-big-and-deep-ai" target="_blank">Crowd Computing for Big and Deep AI</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64091012" target="_blank">智慧型工程管考系統 : 資料分析經驗談</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64091174" target="_blank">音樂資料檢索</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64091204" target="_blank">當語音處理遇上深度學習</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64091657" target="_blank">公私協力的公共服務 - 以資料面詮釋</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64091725" target="_blank">從電腦視覺看人工智慧 : 下一件大事</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64092758" target="_blank">我在智慧交通資料解析的失敗歷程</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/its-not-what-you-say-its-how-you-say-it" target="_blank">It's Not What You Say: It's How You Say It!</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64092823" target="_blank">貓都學得會的手機維修資料分析</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/the-growth-of-a-data-scientist" target="_blank">The Growth of a Data Scientist</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/practical-issues-in-machine-learning" target="_blank">Practical Issues in Machine Learning</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64092909" target="_blank">沒有大數據怎麼辦 ? 會計師事務所的小數據科學</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/feature-engineering-in-machine-learning" target="_blank">Feature Engineering in Machine Learning</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/r-r-64092963" target="_blank">漫談 R 的學習挑戰與 R 語言翻轉教室</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64092976" target="_blank">娛樂產業中的資料科學家 : 談資料科學於線上遊戲與職業運動之應用</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64093015" target="_blank">空間數據分析推動精準新農業</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/ss-64093761" target="_blank">世代之爭爭什麼 ? 談談如何從調查資料挖掘出豐厚的意義</a></li>
<li><a href="http://www.slideshare.net/tw_dsconf/20-64093801" target="_blank">資料視覺化的 20 個小訣竅</a></li>
</ol>
<div>
相關新聞:</div>
<div>
<ol>
<li><a href="http://www.ithome.com.tw/news/107163" target="_blank">陳昇瑋:2016是臺灣資料科學啟動元年,不再只聽,今年真的動起來了</a></li>
<li><a href="http://www.bnext.com.tw/article/view/id/40220" target="_blank">大數據會消失,資料科學不會!你該知道的資料科學第一堂課</a></li>
</ol>
</div>
<div>
其他相關資訊:</div>
<div>
<ol>
<li><a href="https://dsp.hackpad.com/R--zX1JuCX7uEF" target="_blank">R 語言資料工程及探勘實務共筆</a></li>
</ol>
</div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-69362894674054965902016-05-22T16:05:00.000+08:002016-05-23T08:29:39.230+08:00一天上手SparkRSpark是近幾年在數據分析領域非常火紅的技術,繼上一次「<a href="http://dataology.blogspot.tw/2015/10/pythonspark.html">透過Python與Spark做氣象大數據分析</a>」的文章,筆者這次希望分享的是原生Spark 在 R上面的改變,從Spark 1.4版開始對R有較完整的支持,而今年推出的Spark 1.6版上面,可以看到Spark 已經特別為R做了一個專屬的核心,這個核心也只有R才有,其目標就是希望把過去R累積的運算套件無痛移植到Spark上。<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-RnW1X2zqLlM/V0FhTexcPAI/AAAAAAAATAM/ENetguFDvfoXu8BwfH8s80bq2rHpxXRUQCLcB/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="238" src="https://1.bp.blogspot.com/-RnW1X2zqLlM/V0FhTexcPAI/AAAAAAAATAM/ENetguFDvfoXu8BwfH8s80bq2rHpxXRUQCLcB/s400/1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: , serif;">圖1:SparkR架構(<a href="https://people.csail.mit.edu/matei/papers/2016/sigmod_sparkr.pdf" target="_blank">來源</a>)</span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "新細明體" , "serif"; font-size: 12.0pt;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: , serif;">從「<a href="https://people.csail.mit.edu/matei/papers/2016/sigmod_sparkr.pdf" target="_blank">SparkR: Scaling R Programs with Spark</a>」這篇期刊文章上可以觀察到,圖1中的R的Spark Context透過R-JVM跟Java Spark Context做溝通,將工作分散到每個有R的Worker上面,透過Spark Executor進行運算,其實有Hadoop MR的味道。</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: , serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: , serif;">但重點是效率變得更好,程式碼變得更好寫,從效率來看,圖2中SparkR的DataFrame效率跟Scala、Python其實也差不多了,而且做Cache會更快。</span></div>
<div style="text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-Z9GiYLfYI3A/V0Fh0ot0FBI/AAAAAAAATAQ/6hY_23TrhhEmTVpPyD07Sg0oB0FEwo78ACLcB/s1600/2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="145" src="https://2.bp.blogspot.com/-Z9GiYLfYI3A/V0Fh0ot0FBI/AAAAAAAATAQ/6hY_23TrhhEmTVpPyD07Sg0oB0FEwo78ACLcB/s400/2.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
圖2:DataFrame的效率比較(<a href="https://people.csail.mit.edu/matei/papers/2016/sigmod_sparkr.pdf" target="_blank">來源</a>)</div>
<br />
另外一個重點是寫R的程式碼邏輯不需變換,因此寫R的資料科學家可以更輕鬆的轉移過去的R程式碼,另外2015年從SparkR 1.4開始已提供<a href="https://spark.apache.org/docs/1.4.0/api/R/" target="_blank">66種函式</a>,到Spark1.5提供<a href="https://spark.apache.org/docs/1.5.0/api/R/" target="_blank">197個函式</a>,到今年的SparkR 1.6.x版本已提供<a href="https://spark.apache.org/docs/1.6.0/api/R/" target="_blank">225種函式</a>,這種增長速度是非常驚人的,因為SparkR 1.4 是去年2015年6月發表,距離筆者寫這篇文章的時間才剛剛準備要滿一年,因此SparkR的發展非常的快速。<br />
<br />
接下來我們就來用Windows平台先快快的體驗一下SparkR的威力吧,首先到<a href="http://spark.apache.org/downloads.html" target="_blank">Spark官網</a>上去下載Spark,透過下圖的點選,下載spark-1.6.1-bin-hadoop2.6.tgz版本。<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-MM6lcwB_bc4/V0FidWOEOdI/AAAAAAAATAY/_N9hP0UGIkkeJOvFHX96cvfzyeQzAX6MwCLcB/s1600/3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="136" src="https://1.bp.blogspot.com/-MM6lcwB_bc4/V0FidWOEOdI/AAAAAAAATAY/_N9hP0UGIkkeJOvFHX96cvfzyeQzAX6MwCLcB/s400/3.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
下載完成後直接解壓縮到您想要放的位置,像筆者是放在C槽區,並把名字變更為Spark。接下來打開<a href="https://www.rstudio.com/" target="_blank">RStudio</a>、<a href="https://www.r-project.org/" target="_blank">R</a>、<a href="https://mran.microsoft.com/open/" target="_blank">Microsoft R Open</a>都可以,這篇筆者以RStudio做示範。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both;">
首先我們進行環境的設定</div>
<div class="separator" style="clear: both;">
<br /></div>
<code>
Sys.setenv(SPARK_HOME = "C:\\spark") #其中C:\\Spark是您的Spark路徑與目錄名稱
library(SparkR, lib.loc = c(file.path(Sys.getenv("SPARK_HOME"), "R", "lib")))
library(SparkR) </code><br />
接下來我們就可以開始使用SparkR,我們先使用下面兩行指令做起手式。<br />
<br />
<code>
sc = sparkR.init();sqlContext = sparkRSQL.init(sc)
</code>
<br />
其中sc的第一行可以設定Spark的初始狀態,如:要給Spark的記憶體,詳細可參考SparkR的<a href="https://spark.apache.org/docs/1.6.1/api/R/sparkR.init.html">SparkR.init文件</a>。第二行就是前面筆者提到的SparkContext,主要是從JavaSparkContext做初始化的動作。
<br />
<br />
接下來我們來玩玩經典的iris例子,首先先把iris的欄位名稱做修改,因為Spark裡面很多「.」的函式,避免出錯,我們先把欄位名稱以「_」取代。
<br />
<br />
<code>
newiris = iris;names(newiris) = c("Sepal_Length","Sepal_Width","Petal_Length","Petal_Width","Species")
</code>
<br />
接下來我們產生Spark超強的DataFrame,這對R使用者不陌生,只是這次是使用Spark去產生DataFrame。
<br />
<br />
<code>
df = createDataFrame(sqlContext, newiris)
</code>
<br />
接下來我們可以使用head或showDF函式觀察資料。
<br />
<br />
<code>
head(df, 5)
showDF(df, 5)
</code>
<br />
接下來我們可以透過printSchema去觀察一下資料的架構。
<br />
<br />
<code>
printSchema(df)
</code>
<br />
接著我們可以透過head與select去挑選某個欄位的資料,如挑選Sepal_Length。
<br />
<br />
<code>
head(select(df, df$Sepal_Length))
head(select(df, df$Sepal_Length),10) #挑選10筆
</code>
<br />
假設筆者想挑選Sepal_Length大於5.4的資料,可以使用head與filter這兩個函式,這兩個函式用起來有沒有覺得很像R的dplyr呢? 相信R使用者很快就能夠上手SparkR。
<br />
<br />
<code>
head(filter(df, df$Sepal_Length > 5.4))
</code>
<br />
如果dplyr您夠了解,那接下來如果我們想計算Sepal_Length的做簡單的Aggregation,就可以使用下面的指令。
<br />
<br />
<code>
sepal_counts = summarize(groupBy(df, df$Sepal_Length), count = n(df$Sepal_Length))
</code>
<br />
當然如果要做廣義線性模型(GLM),SparkR的做法跟原生的R做法是完全一模一樣。
<br />
<br />
<code>
model = glm(Sepal_Length ~., data = df, family = "gaussian");summary(model)
</code>
<br />
如果還想知道其他的功能,請直接拜訪SparkR的<a href="http://spark.apache.org/docs/latest/api/R/index.html">文件</a><br />
<br />
從本篇文章中我們可以快速掌握SparkR的使用,同時也了解到目前SparkR不僅支援資料選擇(Selection)、資料過濾(Filtering)、資料聚合(Aggregation),更已經開始支援Spark MLlib可以進行分散式的機器學習。因此我們可以期待未來R的資料科學家在分散式運算上可以透過SparkR更暢通無阻。<br />
<br />
當然實務上真的全部資料都要使用SparkR嗎? 不一定,R在記憶體運算(In-memory computing)的部分也有很強的處理能力,因此有時候單機的記憶體夠多,像是<a href="https://mran.microsoft.com/download/">MRO</a> + <a href="https://software.intel.com/en-us/intel-mkl">Math Kernel Library</a>,或是寫法稍微改一下(如矩陣寫法),基本上也無須要動用SparkR,雖然如此,在分散式的環境下SparkR還是有它能發展的舞台,所以讓我們繼續期待下去。<br />
<div>
<br /></div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-29823218639993412552016-03-23T16:03:00.000+08:002016-03-23T16:03:01.710+08:00雲端機器學習上手篇-以非監督式學習演算法為例由於筆者對於數據分析都非常有興趣,也因為工作關係需要接觸到各種不同的雲端數據分析工具,繼上次筆者分享過「<a href="http://dataology.blogspot.tw/2015/10/pythonspark.html">透過Python與Spark做氣象大數據分析</a>」是屬於IBM Bluemix的服務,這次筆者要分享的是關於2015年2月微軟推出的<a href="https://www.youtube.com/watch?v=SJtNJepz-pM">Azure Machine Learning</a>工具,有興趣的朋友可以從<a href="https://asiasoutheast.studio.azureml.net/?selectAccess=true&o=1">這裡</a>選擇「Guest Access」快速進行試用。或是申請一個<a href="https://azure.microsoft.com/zh-tw/free/">帳號</a>,依照筆者的步驟執行,進行Microsoft Azure Machine Learning(MAML)的體驗。<br />
<br />
過去在實務上經常發現要做一個數學模型與系統整合一起應用,是非常困難的一件事情,通常作法是需要先請統計學家設計好數學模型並驗證後,再交由軟體工程師進行系統整合,然後再依照市場需求做調整,再這樣反覆的過程中,統計學家改模型,軟體工程師調整系統,人力物力都消耗非常多。<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-_MKTTkXGw6k/Vt0LtokNXgI/AAAAAAAAS5U/GWeyt4lEiUs/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="139" src="https://4.bp.blogspot.com/-_MKTTkXGw6k/Vt0LtokNXgI/AAAAAAAAS5U/GWeyt4lEiUs/s320/1.png" width="320" /></a></div>
<br />
<br />
在微軟這套Azure Machine Learning有幾個讓筆者覺得很方便的地方,筆者條列如下<br />
<br />
<ol>
<li>應用模型即刻上線</li>
<li>多種現成應用選擇</li>
<li>採用拖拉互動介面</li>
</ol>
<div>
第一「應用模型即可上線」就是機器學習後的模型可反覆利用之外,還可快速佈署到雲端上,以筆者為例,要預測汽車價格,可在二十分鐘內完成模型佈署。第二「多種現成應用選擇」在Azure上有許多使用者分享了數學模型可以馬上使用,像是<a href="https://azure.microsoft.com/en-us/documentation/articles/machine-learning-create-experiment/" target="_blank">預測汽車價格</a>、<a href="https://gallery.cortanaanalytics.com/Experiment/Movie-Recommender-Models-created-using-R-1" target="_blank">電影推薦</a>、<a href="https://gallery.cortanaanalytics.com/?fromlegacydomain=1" target="_blank">更多</a>......等。第三是「採用拖拉互動介面」介面使用容易,類似SPSS、SAS的概念,都是採用資料探勘的流程來進行資料分析。</div>
<div>
<br /></div>
<br />
接下來在本次的體驗中,我們採用最經典的<a href="https://zh.wikipedia.org/wiki/%E5%AE%89%E5%BE%B7%E6%A3%AE%E9%B8%A2%E5%B0%BE%E8%8A%B1%E5%8D%89%E6%95%B0%E6%8D%AE%E9%9B%86" target="_blank">安德森鳶尾花卉數據集(Anderson's Iris data set)為例</a>,並使用非監督式學習(Unsupervised Learning)方法中的K-means演算法來進行實作,以快速體驗雲端機器學習的便利性。<br />
<br />
首先大家須先登入<a href="https://azure.microsoft.com/zh-tw/" target="_blank">Microsoft Azure</a>的介面,目前Azure新版的介面只要使用到Azure Machine Learning都會跳轉到舊的介面,因此就參考筆者以下的畫面,照著操作即可。<br />
<br />
首先選擇機器學習服務。<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
選擇「新增」,並點選「快速建立」,我們可以在工作區名稱上,建立自己的工作區名稱,像是ML這樣的名稱。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-C77Lz3iP5y0/Vt0Jt4er7EI/AAAAAAAAS0s/K67Bjli5RDQ/s1600/02.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://3.bp.blogspot.com/-C77Lz3iP5y0/Vt0Jt4er7EI/AAAAAAAAS0s/K67Bjli5RDQ/s640/02.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-4R1A41UFKK4/Vt0Jt3q-vuI/AAAAAAAAS00/5SMuxeLbwXc/s1600/03.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="264" src="https://3.bp.blogspot.com/-4R1A41UFKK4/Vt0Jt3q-vuI/AAAAAAAAS00/5SMuxeLbwXc/s640/03.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著進入工作區之後,點選下方的「在Studio中開啟」的選項,進入MAML的使用介面。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-oaGC-Mt8ecA/Vt0JueO9BrI/AAAAAAAAS04/5yC-EA6nppg/s1600/04.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="219" src="https://3.bp.blogspot.com/-oaGC-Mt8ecA/Vt0JueO9BrI/AAAAAAAAS04/5yC-EA6nppg/s320/04.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們必須點選「DATASETS」去上傳資料集,將資料做上船的動作,我們採用<a href="https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data" target="_blank">這裡</a>的資料,這是一份相當經典的示範資料集,只要到Google搜尋iris csv就有許多參考資料,資料中有150筆資料與5種資料欄位,下載之後我們點選「NEW」如下圖所示。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-pctqWvRAcTc/Vt0JulO1LmI/AAAAAAAAS08/PmZsJ0EQfZY/s1600/05.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://1.bp.blogspot.com/-pctqWvRAcTc/Vt0JulO1LmI/AAAAAAAAS08/PmZsJ0EQfZY/s400/05.png" width="285" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著點選「FROM LOCAL FILE」將資料進行上傳的動作。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-tgUXgWKdKzM/Vt0Juuz1aLI/AAAAAAAAS1A/e5LGoewVOa8/s1600/06.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="210" src="https://2.bp.blogspot.com/-tgUXgWKdKzM/Vt0Juuz1aLI/AAAAAAAAS1A/e5LGoewVOa8/s400/06.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著在介面上面直接「選擇檔案」,將檔案選好之後,點選右下角的勾勾,就可以進行上傳。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-wWyi8fcK7og/Vt0JvA7t6yI/AAAAAAAAS1E/pqp_N5QmEFs/s1600/07.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="298" src="https://1.bp.blogspot.com/-wWyi8fcK7og/Vt0JvA7t6yI/AAAAAAAAS1E/pqp_N5QmEFs/s320/07.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著點選「EXPERIMENTS」,再點選下方的「NEW」。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-UshfzAIaLgk/Vt0JvNqGkDI/AAAAAAAAS1I/8LFy7SsdEVE/s1600/08.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://1.bp.blogspot.com/-UshfzAIaLgk/Vt0JvNqGkDI/AAAAAAAAS1I/8LFy7SsdEVE/s400/08.png" width="302" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們會看到許多Machine Learning Samples,不過我們今天要自己做一個,所以點選「Blank Experiment」可以產生新的實驗環境。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-YXIn2XAgQ9c/Vt0JvTUTLeI/AAAAAAAAS1M/S2T0fisMpMg/s1600/09.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://1.bp.blogspot.com/-YXIn2XAgQ9c/Vt0JvTUTLeI/AAAAAAAAS1M/S2T0fisMpMg/s400/09.png" width="313" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著看到工作區後<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="35ccd115-5a1d-4698-a5af-e576ffde6468" id="3091902c-63bf-4072-b432-3bd641f974cf">,</gs>我們可以在下圖紅色框中輸入這個專案的名稱<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="35ccd115-5a1d-4698-a5af-e576ffde6468" id="6da25dbf-8fb7-4015-8212-6deba16f80b7">,</gs><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="35ccd115-5a1d-4698-a5af-e576ffde6468" id="987159e6-a40b-4e1a-8d62-68ca9fa59d36">像是筆者輸入「My</gs> First Azure Machine Learning Case<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="35ccd115-5a1d-4698-a5af-e576ffde6468" id="0b9b513c-6e84-48f4-8cd4-6b82c8f78c69">」</gs><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="35ccd115-5a1d-4698-a5af-e576ffde6468" id="ecefb88e-8652-4656-80dc-8a74891a31aa">,接著就可以開始實作囉!</gs></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-zOJ3PW_RIzg/Vt0JwYc4PiI/AAAAAAAAS1Q/tRENrZPlslE/s1600/10.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="250" src="https://3.bp.blogspot.com/-zOJ3PW_RIzg/Vt0JwYc4PiI/AAAAAAAAS1Q/tRENrZPlslE/s400/10.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
首先我們從「Saved Datasets」裡面的「My Datasets」將剛剛上傳的資料集「iris.csv」整個拖曳到右邊的工作區。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-MaC9E5Zmb8U/Vt0JwuleT-I/AAAAAAAAS1U/vkknkH5w9U8/s1600/11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="344" src="https://4.bp.blogspot.com/-MaC9E5Zmb8U/Vt0JwuleT-I/AAAAAAAAS1U/vkknkH5w9U8/s640/11.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接下來我們可以很快的對這個資料集做資料視覺化(Visualization),這是筆者覺得滿方便的地方,使用者只要一個右鍵,就可以對資料集做初步的了解。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-JQnTPD4tIEE/Vt0Jw-7fEMI/AAAAAAAAS1Y/YD5GerEr6Ic/s1600/12.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="172" src="https://4.bp.blogspot.com/-JQnTPD4tIEE/Vt0Jw-7fEMI/AAAAAAAAS1Y/YD5GerEr6Ic/s320/12.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
這個簡易的資料視覺化目前可以透過盒鬚圖(Box-plot)、長條圖(Bar Chart)、平均數(Mean)、中位數(Median)、最小值(Min)、最大值(Max)、標準差(Standard Deviation)......等統計資訊去觀察資料的狀況。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-J_Qm3fLaT4w/Vt0JxZmzGLI/AAAAAAAAS1c/MAH-5A6ZwCY/s1600/13.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="254" src="https://1.bp.blogspot.com/-J_Qm3fLaT4w/Vt0JxZmzGLI/AAAAAAAAS1c/MAH-5A6ZwCY/s640/13.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著原始iris資料集裡面包含萼片長(Sepal length)、萼片寬(Sepal width)、花瓣長(Petal length)、花瓣寬(Petal width)、種類(Species)共五個欄位,我們先拿掉種類(Species)的欄位,這個欄位並不適合放在這個學習模型中,因此我們在下圖左邊紅框中搜尋「Project Columns」將他拖曳到工作區。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-q9MV8UxmboI/Vt0JyIylNWI/AAAAAAAAS1g/FZyRvVeZAD0/s1600/14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="356" src="https://4.bp.blogspot.com/-q9MV8UxmboI/Vt0JyIylNWI/AAAAAAAAS1g/FZyRvVeZAD0/s640/14.png" width="640" /></a></div>
<br />
接著點選iris資料集的圓點,拖曳到「<span style="text-align: center;">Project Columns」的圓</span><span style="text-align: center;">點,讓兩個功能做一個連線,如</span>下圖所示<span style="text-align: center;">。</span><br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-1CPahKqsoow/Vt0Jz9ge2kI/AAAAAAAAS1o/k3GaHm_GtDw/s1600/15.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="346" src="https://4.bp.blogspot.com/-1CPahKqsoow/Vt0Jz9ge2kI/AAAAAAAAS1o/k3GaHm_GtDw/s640/15.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著點選「Project Columns」,並點選右邊紅框的「Launch column selector」按鈕。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-XeGmcDlcJCw/Vt0Jz_Pn2xI/AAAAAAAAS1k/taWS3oB_w4s/s1600/16.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="324" src="https://4.bp.blogspot.com/-XeGmcDlcJCw/Vt0Jz_Pn2xI/AAAAAAAAS1k/taWS3oB_w4s/s640/16.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著選擇「WITH RULES」→「ALL COLUMNS」→「Exclude」將「Class」欄位做排除的動作,如下圖示意。</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-MLvRZg1Mn4g/Vt0J0g_XSKI/AAAAAAAAS1w/ShjElMev9IQ/s1600/18.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="314" src="https://4.bp.blogspot.com/-MLvRZg1Mn4g/Vt0J0g_XSKI/AAAAAAAAS1w/ShjElMev9IQ/s640/18.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
因為一般來說,我們必須將資料分割成訓練集、測試集才能進行實驗,因此我們必須進行資料切割,這邊使用「Split Data 」,我們找到他之後,將他拖曳到右邊工作區。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-lP-YN8CnD1Q/Vt0J1NniBRI/AAAAAAAAS10/9yzWHLgm-uM/s1600/19.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://3.bp.blogspot.com/-lP-YN8CnD1Q/Vt0J1NniBRI/AAAAAAAAS10/9yzWHLgm-uM/s640/19.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著一樣與Project Columns做連線。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-ruBsPUd9hu0/Vt0J1c7ANOI/AAAAAAAAS14/fLTJDLdmZcU/s1600/20.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="316" src="https://3.bp.blogspot.com/-ruBsPUd9hu0/Vt0J1c7ANOI/AAAAAAAAS14/fLTJDLdmZcU/s640/20.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們點選「Split Data」的功能,輸入分割的比例,如下圖紅框所示,輸入0.7表示分割成訓練集70%,測試集30%,如果要分6/4就要輸入0.6,以此類推。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-Nwv8AusO6ug/Vt0J1X-NZEI/AAAAAAAAS18/l4UWeO8k6E4/s1600/21.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="324" src="https://3.bp.blogspot.com/-Nwv8AusO6ug/Vt0J1X-NZEI/AAAAAAAAS18/l4UWeO8k6E4/s640/21.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們要有一個控制模型輸出的地方,找到「Train Clustering Model」之後一樣拖曳到右邊工作區。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-xr6A8Jca6Qw/Vt0J1piR_7I/AAAAAAAAS2A/fx0jKrnzyr0/s1600/22.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://1.bp.blogspot.com/-xr6A8Jca6Qw/Vt0J1piR_7I/AAAAAAAAS2A/fx0jKrnzyr0/s320/22.png" width="265" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們將「Split Data」左下的訓練集(70%)拉到「Train Clustering Model」的右上角的圓點上,表示將訓練集放進去。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-JoY_8bcKW5A/Vt0J2Fj_EYI/AAAAAAAAS2I/UhmkQIa4TDQ/s1600/24.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="288" src="https://4.bp.blogspot.com/-JoY_8bcKW5A/Vt0J2Fj_EYI/AAAAAAAAS2I/UhmkQIa4TDQ/s640/24.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們在Machine Learning裡面找到「<a href="https://zh.wikipedia.org/wiki/K-%E5%B9%B3%E5%9D%87%E7%AE%97%E6%B3%95" target="_blank">K-mean Clustering</a>」這就是今天我們提到的非監督式學習(Unsupervised Learning)演算法的一種,我們將他拖曳到工作區。</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-0gGR4Iage18/Vt0J2UX9-kI/AAAAAAAAS2Q/a8FfQwitrWo/s1600/26.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="324" src="https://1.bp.blogspot.com/-0gGR4Iage18/Vt0J2UX9-kI/AAAAAAAAS2Q/a8FfQwitrWo/s640/26.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
將「K-mean Clustering」的圓點連線到「Train Clustering Model」的左上角圓點,表示我們要使用該演算法建立模型。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-ZVkEoqZwEnM/Vt0J2_6bLDI/AAAAAAAAS2U/_SuHNDBdKaw/s1600/27.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="350" src="https://1.bp.blogspot.com/-ZVkEoqZwEnM/Vt0J2_6bLDI/AAAAAAAAS2U/_SuHNDBdKaw/s640/27.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
因為集群計算之後,都需要設定集群結果給資料集,所以我們要使用「Assign to clusters」這個功能配置模型預測結果。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-ZUSX3IafF7A/Vt0J288CIFI/AAAAAAAAS2Y/CHjbM8suL0w/s1600/28.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://2.bp.blogspot.com/-ZUSX3IafF7A/Vt0J288CIFI/AAAAAAAAS2Y/CHjbM8suL0w/s640/28.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著將「Train Clustering Model」左下角的圓點,與「Assign to Clusters」上方的圓點進行相連。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-sQjgsU2sQ9s/Vt0J4dDZk0I/AAAAAAAAS2c/H1lVCv7OJZk/s1600/29.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://2.bp.blogspot.com/-sQjgsU2sQ9s/Vt0J4dDZk0I/AAAAAAAAS2c/H1lVCv7OJZk/s640/29.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
最後我們要再使用「Project Columns」將運算結果輸出,所以一樣找到「Project Columns」然後拖曳到工作區。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-BqWftJhqNyg/Vt0J5FqOsGI/AAAAAAAAS2g/km-Vcs6m-gg/s1600/30.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="324" src="https://4.bp.blogspot.com/-BqWftJhqNyg/Vt0J5FqOsGI/AAAAAAAAS2g/km-Vcs6m-gg/s640/30.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們會看到很多個元件上都有紅色驚嘆號,因為我們都還沒幫他設定資料的輸入或輸出。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-0yBmEgvSmhc/Vt0J5X-j1mI/AAAAAAAAS2k/nKEGnLc_TRk/s1600/31.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="350" src="https://3.bp.blogspot.com/-0yBmEgvSmhc/Vt0J5X-j1mI/AAAAAAAAS2k/nKEGnLc_TRk/s640/31.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
不過沒關係,我們繼續將「Project Columns」再拖一個出來,將「Train Clustering Model」的右下角圓點與它做連線,這個動作主要是將原始的資料做輸出,不做也能完成實驗。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-ew9WOqyerjQ/Vt0J5udUjUI/AAAAAAAAS2o/b099L-5c95g/s1600/32.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="346" src="https://4.bp.blogspot.com/-ew9WOqyerjQ/Vt0J5udUjUI/AAAAAAAAS2o/b099L-5c95g/s640/32.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著如上面所述,我們將「Train Clustering Model」與它做一個連線,如下圖所示</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-m-Ns-zBKoho/Vt0J5_tFJDI/AAAAAAAAS2s/xDzAJzp6HX8/s1600/33.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="282" src="https://2.bp.blogspot.com/-m-Ns-zBKoho/Vt0J5_tFJDI/AAAAAAAAS2s/xDzAJzp6HX8/s640/33.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們要開始針對這些放上去的元件做參數設定,我們先從「K-mean Clustering」開始,我們將他的中心數(K)設定為3個,也就是要分成3群。</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-_3rE2uN_bco/Vt0J7310zzI/AAAAAAAAS2w/2YGKrN0hEC0/s1600/34.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="323" src="https://4.bp.blogspot.com/-_3rE2uN_bco/Vt0J7310zzI/AAAAAAAAS2w/2YGKrN0hEC0/s640/34.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們點選「Train Clustering Model」的「Launch column selector」按鈕。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-GZZ3KLHXs70/Vt0J81QKqbI/AAAAAAAAS24/ApmZ_abpCMA/s1600/35.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="322" src="https://1.bp.blogspot.com/-GZZ3KLHXs70/Vt0J81QKqbI/AAAAAAAAS24/ApmZ_abpCMA/s640/35.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
將他的輸入設定成包含萼片長(Sepal length)、萼片寬(Sepal width)、花瓣長(Petal length)、花瓣寬(Petal width)四個欄位,如下圖所示。</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-2I2AEfmcQT8/Vt0J8ngIQiI/AAAAAAAAS20/JwbxvnAV5xE/s1600/36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="307" src="https://2.bp.blogspot.com/-2I2AEfmcQT8/Vt0J8ngIQiI/AAAAAAAAS20/JwbxvnAV5xE/s640/36.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們將「Assign to Clusters」右上角的圓點與「Split Data」右下角的圓點,也就是測試資料集做連接,如下圖指標所示。</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-3dMiJPHV2KE/Vt0J9YjCnkI/AAAAAAAAS3A/eQYTNCbjMWA/s1600/38.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="334" src="https://2.bp.blogspot.com/-3dMiJPHV2KE/Vt0J9YjCnkI/AAAAAAAAS3A/eQYTNCbjMWA/s640/38.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們選擇「Assign to Clusters」的「Launch column selector」按鈕,設定要使用的欄位。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-xLs9q6TOzjs/Vt0J9nFOB8I/AAAAAAAAS3E/40hjRrT-Xhc/s1600/39.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="331" src="https://2.bp.blogspot.com/-xLs9q6TOzjs/Vt0J9nFOB8I/AAAAAAAAS3E/40hjRrT-Xhc/s640/39.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
一樣包含萼片長(Sepal length)、萼片寬(Sepal width)、花瓣長(Petal length)、花瓣寬(Petal width)四個欄位,如下圖所示。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-q-FYcpLbUZc/Vt0J93saWvI/AAAAAAAAS3I/U2JHPuNSQag/s1600/40.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="314" src="https://2.bp.blogspot.com/-q-FYcpLbUZc/Vt0J93saWvI/AAAAAAAAS3I/U2JHPuNSQag/s640/40.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們對最後一個「Project Columns」做個設定。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-c5Z6U6dCh78/Vt0J-FcWV7I/AAAAAAAAS3M/PZunQMmub5M/s1600/41.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="332" src="https://2.bp.blogspot.com/-c5Z6U6dCh78/Vt0J-FcWV7I/AAAAAAAAS3M/PZunQMmub5M/s640/41.png" width="640" /></a></div>
<br />
<div style="text-align: left;">
<span style="text-align: center;">注意這邊除了選擇包含萼片長(Sepal length)、萼片寬(Sepal width)、花瓣長(Petal length)、花瓣寬(Petal width)四個欄位之外,還要再選擇「Assignment」,如下圖所示。</span></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-RRA1EAqR0uw/Vt0J-bEPSSI/AAAAAAAAS3Q/kxi6x-61rzU/s1600/42.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://3.bp.blogspot.com/-RRA1EAqR0uw/Vt0J-bEPSSI/AAAAAAAAS3Q/kxi6x-61rzU/s640/42.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著點選「SAVE」,再點選「RUN」。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-MI8cwnRo8Sc/Vt0J-T9J1KI/AAAAAAAAS3Y/im3O2VOrqeM/s1600/43.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="392" src="https://3.bp.blogspot.com/-MI8cwnRo8Sc/Vt0J-T9J1KI/AAAAAAAAS3Y/im3O2VOrqeM/s640/43.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著會看到運行時間,因為資料不多,所以在幾秒內即可完成。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-XIVRTBgmLc0/Vt0J-9-9GqI/AAAAAAAAS3U/z5111nWG0Hg/s1600/44.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="346" src="https://4.bp.blogspot.com/-XIVRTBgmLc0/Vt0J-9-9GqI/AAAAAAAAS3U/z5111nWG0Hg/s640/44.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著到這裡會出現下面錯誤,就是「Project Columns」找不到「Assignment」的問題,如下圖所示。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/--bzDsGVKGTM/Vt0J_gS88zI/AAAAAAAAS3c/PLFb92j9L5U/s1600/45.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="340" src="https://3.bp.blogspot.com/--bzDsGVKGTM/Vt0J_gS88zI/AAAAAAAAS3c/PLFb92j9L5U/s640/45.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
不過沒關係,我們只要到「Project Columns」裡面,再點選「Launch column selector」。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-b285EHyKfGM/Vt0J_0uUihI/AAAAAAAAS3g/NbVe05xZr84/s1600/46.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="318" src="https://3.bp.blogspot.com/-b285EHyKfGM/Vt0J_0uUihI/AAAAAAAAS3g/NbVe05xZr84/s640/46.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
然後把剛剛選的「Assignment」刪掉再選一次即可,筆者推敲問題的原因是,模型還沒計算,所以第一次不會有「Assignment」的結果,所以要再設定一次。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-GoYsWSXVjJc/Vt0J_9LaxiI/AAAAAAAAS3k/bWPpMMAcu24/s1600/47.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="322" src="https://3.bp.blogspot.com/-GoYsWSXVjJc/Vt0J_9LaxiI/AAAAAAAAS3k/bWPpMMAcu24/s640/47.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著再「SAVE」與「RUN」一次</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-j2It9jOJqW4/Vt0KATaNvXI/AAAAAAAAS3s/_zLyd08L0yc/s1600/48.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="392" src="https://1.bp.blogspot.com/-j2It9jOJqW4/Vt0KATaNvXI/AAAAAAAAS3s/_zLyd08L0yc/s640/48.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們就可以點選「Project Columns」裡面的「Visualize」就可以看到分群的結果。</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-bNDuu-07QAk/Vt0KAkY5xYI/AAAAAAAAS3w/9sTQxSJivbM/s1600/50.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="396" src="https://3.bp.blogspot.com/-bNDuu-07QAk/Vt0KAkY5xYI/AAAAAAAAS3w/9sTQxSJivbM/s640/50.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
我們可以發現測試集的45筆資料,多了一個「Assignments」並且裡面分了0、1、2共三個群。</div>
<div class="separator" style="clear: both; text-align: left;">
這就是我們初步的成果,不過還沒結束,接下來我們想把模型應用到系統上。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-Whr4e67S9d4/Vt0KAzgLwfI/AAAAAAAAS30/5vfhMcB8YhU/s1600/51.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="262" src="https://4.bp.blogspot.com/-Whr4e67S9d4/Vt0KAzgLwfI/AAAAAAAAS30/5vfhMcB8YhU/s640/51.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
我們將要進行機器學習與系統整合這塊實作,在「Web Service」裡面,我們拖曳出「Input」。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-Wq0s-woRC8U/Vt0KBPLhhkI/AAAAAAAAS38/S6Zu2uKQ_8I/s1600/52.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="350" src="https://2.bp.blogspot.com/-Wq0s-woRC8U/Vt0KBPLhhkI/AAAAAAAAS38/S6Zu2uKQ_8I/s640/52.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著將「Web service input」 與「Project Columns」做連接,表示新的資料有一部分會從這裡輸入。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-SKTg-m3PCrQ/Vt0KBYAI34I/AAAAAAAAS34/dr79DRNOduY/s1600/53.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="352" src="https://2.bp.blogspot.com/-SKTg-m3PCrQ/Vt0KBYAI34I/AAAAAAAAS34/dr79DRNOduY/s640/53.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們將「Output」也拖曳到工作區,並將「Assign to Clusters」的下方圓點與「Web Service Output」做連接。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-FSC0y0K_63I/Vt0KB8fQZUI/AAAAAAAAS4A/273t5fwq8Aw/s1600/54.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="342" src="https://2.bp.blogspot.com/-FSC0y0K_63I/Vt0KB8fQZUI/AAAAAAAAS4A/273t5fwq8Aw/s640/54.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們點選「SAVE」、「RUN」。</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-l5exkOmGfRQ/Vt0KCKRifnI/AAAAAAAAS4I/xm7_fANdJIw/s1600/56.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="392" src="https://2.bp.blogspot.com/-l5exkOmGfRQ/Vt0KCKRifnI/AAAAAAAAS4I/xm7_fANdJIw/s640/56.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
運算完之後,我們點選「SET UP WEB SERVICE」裡面的「Predictive Web Service」進行系統佈署前的設定。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-pVEENeUJeTQ/Vt0KCpgFJRI/AAAAAAAAS4M/hoP0i2fs40o/s1600/57.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="366" src="https://3.bp.blogspot.com/-pVEENeUJeTQ/Vt0KCpgFJRI/AAAAAAAAS4M/hoP0i2fs40o/s640/57.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著可以看到Azure自動將我們的模型與系統做流程調整。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-sOBEd84B7PA/Vt0KDIzt8LI/AAAAAAAAS4Q/0ZStUurRI7c/s1600/58.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://4.bp.blogspot.com/-sOBEd84B7PA/Vt0KDIzt8LI/AAAAAAAAS4Q/0ZStUurRI7c/s640/58.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
設定完之後,再做一次「SAVE」、「RUN」,將系統模型做儲存。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-BsUIx5CnuSw/Vt0KDMBwjuI/AAAAAAAAS4U/GOywEt2tMlg/s1600/59.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="396" src="https://3.bp.blogspot.com/-BsUIx5CnuSw/Vt0KDMBwjuI/AAAAAAAAS4U/GOywEt2tMlg/s640/59.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們再點選一次「SET UP WEB SERVICE」就可以讓我們的演算法轉成API給系統使用,API全名為應用程式介面(Application Programming Interface),通常就是系統與系統間做傳輸共通標準,有了這個標準後,工程師不管寫甚麼系統,都可以直接透過API跟模型做連接,這個就是筆者覺得很方便的地方,因為這樣資料科學家就可以專注做模型即可。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-Vi9dmT305ag/Vt0KDkV9GJI/AAAAAAAAS4Y/BtWocl1Qw10/s1600/60.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="388" src="https://2.bp.blogspot.com/-Vi9dmT305ag/Vt0KDkV9GJI/AAAAAAAAS4Y/BtWocl1Qw10/s640/60.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
獲得API之後我們可以做一個簡單測試,點選「Test」。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-iPCy7aRlXJo/Vt0KD-5IntI/AAAAAAAAS4c/-i6HnlZKkDg/s1600/61.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="179" src="https://2.bp.blogspot.com/-iPCy7aRlXJo/Vt0KD-5IntI/AAAAAAAAS4c/-i6HnlZKkDg/s320/61.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
我們嘗試在框框中輸入萼片長(Sepal length)、萼片寬(Sepal width)、花瓣長(Petal length)、花瓣寬(Petal width)四個欄位的新數字,也就是我們想要透過我們的模型去預測它的分群。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-Fi1mtGNPfhs/Vt0KD_rux_I/AAAAAAAAS4g/egp98bY3R2M/s1600/62.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="282" src="https://3.bp.blogspot.com/-Fi1mtGNPfhs/Vt0KD_rux_I/AAAAAAAAS4g/egp98bY3R2M/s320/62.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
點選確認之後,我們就會在下方看到預測結果,筆者依次輸入1.2、1.5、1.7、1.9,他就被分到第1群(我們共有0,1,2三群)。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-sYm5orR-Dfk/Vt0KEEwmTRI/AAAAAAAAS4k/fk42ByYbwSs/s1600/63.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="348" src="https://1.bp.blogspot.com/-sYm5orR-Dfk/Vt0KEEwmTRI/AAAAAAAAS4k/fk42ByYbwSs/s640/63.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們想要佈署到系統上,Azure很貼心的準備了C#、R、Python樣本程式碼,我們點選下圖左邊紅框的選項。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-O94as7MkAik/Vt0KEXu4kHI/AAAAAAAAS4o/ERH8HCyIH7Q/s1600/64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="504" src="https://2.bp.blogspot.com/-O94as7MkAik/Vt0KEXu4kHI/AAAAAAAAS4o/ERH8HCyIH7Q/s640/64.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
除了一些API使用說明之外,這裡就附上了三種語言的樣本檔案。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-Af5jDNRvcf0/Vt0KEqTlYbI/AAAAAAAAS4s/-QWvDD-zJg0/s1600/65.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://1.bp.blogspot.com/-Af5jDNRvcf0/Vt0KEqTlYbI/AAAAAAAAS4s/-QWvDD-zJg0/s640/65.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
在每個程式碼當中,都需要加入api_key才能使用。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-sQcjTULfJw4/Vt0KEzKLv_I/AAAAAAAAS4w/K3H7gXOuGbw/s1600/66.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="258" src="https://3.bp.blogspot.com/-sQcjTULfJw4/Vt0KEzKLv_I/AAAAAAAAS4w/K3H7gXOuGbw/s640/66.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
API KEY的內容可以參考筆者下方紅框中的資料,每個模型都有屬於自己的API KEY,所以不用拷貝筆者的。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-jBUZH-UHqlU/Vt0KFXLEA7I/AAAAAAAAS40/zSEYCpL1fjQ/s1600/67.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="354" src="https://3.bp.blogspot.com/-jBUZH-UHqlU/Vt0KFXLEA7I/AAAAAAAAS40/zSEYCpL1fjQ/s640/67.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著做了替換之後,如果您是像筆者是寫Python 3,那就必須再做一些程式碼調整,如下圖紅框所示。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-MBJTP69sSww/Vt0KFZcyOmI/AAAAAAAAS44/pu_ckrmVdog/s1600/68.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="182" src="https://3.bp.blogspot.com/-MBJTP69sSww/Vt0KFZcyOmI/AAAAAAAAS44/pu_ckrmVdog/s640/68.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-lA-XTqS7i4Q/Vt0KFucSXSI/AAAAAAAAS48/9V0zwr-khqk/s1600/69.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="282" src="https://2.bp.blogspot.com/-lA-XTqS7i4Q/Vt0KFucSXSI/AAAAAAAAS48/9V0zwr-khqk/s640/69.png" width="640" /></a></div>
接著我們可以在Values裡面輸入我們想要預測的數字,執行之後就可以看到相同的結果。<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-jMAMHgeKKME/Vt0KF9djSCI/AAAAAAAAS5A/v7ryCMtqBKo/s1600/70.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="154" src="https://4.bp.blogspot.com/-jMAMHgeKKME/Vt0KF9djSCI/AAAAAAAAS5A/v7ryCMtqBKo/s640/70.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
下圖中,上面是用Test的結果1.2,1.5,1.7,1.8被分到第1群,下面使用Azure樣本python程式碼,一樣也是被分到第1群。</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-kcJ11rFatO8/Vt0KGJSKhyI/AAAAAAAAS5E/KTr3RvsSZAg/s1600/71.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="160" src="https://2.bp.blogspot.com/-kcJ11rFatO8/Vt0KGJSKhyI/AAAAAAAAS5E/KTr3RvsSZAg/s640/71.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
之後就可以把這段程式碼修改並佈署到相關的系統當中。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
以上就是雲端機器學習的上手教學,希望能協助讀者做初步雲端機器學習體驗。</div>
<br />資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-19356170450347879232016-02-22T17:50:00.001+08:002016-03-08T09:33:15.022+08:00透過雲端服務與Node.js打造車聯網大數據分析平台車聯網(Internet of Vehicles)已經是今年在汽車界熱門的議題,從McKinsey去年的<a href="http://apo.org.au/resource/internet-things-mapping-value-beyond-hype" target="_blank">物聯網報告</a>來看,車聯網可以做的事情共有六種,包含「改善汽車的安全與可靠度」、「維修服務推薦」、「加強汽車連結」、「加強客戶體驗」、「提供保險動態價格」、「增加車隊效率與降低成本」,但是做這些事情之前<u><span style="color: #990000;">基礎設施</span></u>就變得非常重要,如果沒有一個好的雲端平台去收集資料、儲存資料、分析資料,就無法與前面提到的服務做整合。<br />
<br />
因此本次教學筆者嘗試使用了一個雲端平台(IBM Bluemix)去進行車聯網的系統實作,希望透過一個簡單的實作,讓有興趣的朋友能掌握技術要領,並實際體驗車聯網的初步成果,如下圖所示。<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<img border="0" height="275" src="https://2.bp.blogspot.com/-FtUl1-5MvAA/VsrYsmcLIuI/AAAAAAAASzw/OH32npEDrDY/s400/22.png" width="400" /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://iotcc.mybluemix.net/" target="_blank">http://iotcc.mybluemix.net/</a></div>
<br />
<br />
首先我們必須要有<a href="https://console.ng.bluemix.net/" target="_blank">Bluemix</a>帳號,進入了平台以後找到,在<a href="https://console.ng.bluemix.net/catalog/" target="_blank">型錄</a>裡面,找到「Internet of Things Platform」,如下圖所示:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-k3ksCybwuqU/Vsq68PD1FWI/AAAAAAAASwY/OpmqANHO4iI/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="140" src="https://1.bp.blogspot.com/-k3ksCybwuqU/Vsq68PD1FWI/AAAAAAAASwY/OpmqANHO4iI/s400/1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著點選他,然後在Service Name上面打入我們的服務名稱,像筆者就打「iotcc」,然後點選「Create」將服務進行啟動,如下圖:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-AoMCfiugK3s/Vsq8DzVtDhI/AAAAAAAASwg/NZJ5XR8utS8/s1600/2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="341" src="https://4.bp.blogspot.com/-AoMCfiugK3s/Vsq8DzVtDhI/AAAAAAAASwg/NZJ5XR8utS8/s400/2.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
之後您回到自己的儀錶板,就可以看到如下的畫面:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-gPrBGFv8Ej0/Vsq8sy3pqeI/AAAAAAAASwo/_utPly_WgUU/s1600/3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="170" src="https://2.bp.blogspot.com/-gPrBGFv8Ej0/Vsq8sy3pqeI/AAAAAAAASwo/_utPly_WgUU/s400/3.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著點選服務名稱,以筆者為例是「iotcc」,點進去以後,再點選「Launch Dashboard」,如下圖:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-7deHv9yyaus/VsrE5lWAL9I/AAAAAAAASw4/oVDMvadr4Bw/s1600/4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="138" src="https://4.bp.blogspot.com/-7deHv9yyaus/VsrE5lWAL9I/AAAAAAAASw4/oVDMvadr4Bw/s400/4.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著點選IBM Watson IoT Platform的「Devices」選項。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-88CEkHyWqNI/VsrFUZ5SlaI/AAAAAAAASw8/lkGe2QyI-9o/s1600/5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="172" src="https://4.bp.blogspot.com/-88CEkHyWqNI/VsrFUZ5SlaI/AAAAAAAASw8/lkGe2QyI-9o/s400/5.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
選擇「Add Device」進行新增裝置。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-_3IAFdRvvOU/VsrF1h82UQI/AAAAAAAASxE/-IzXW4YCfiU/s1600/6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="170" src="https://1.bp.blogspot.com/-_3IAFdRvvOU/VsrF1h82UQI/AAAAAAAASxE/-IzXW4YCfiU/s400/6.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
首先我們必須先建立汽車裝置的裝置類型,因此點選「Create Device Type」進行裝置類型新增。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-vtmgvvmF2QY/VsrGKAEv_nI/AAAAAAAASxI/jLFpJiCg3RY/s1600/7.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="205" src="https://1.bp.blogspot.com/-vtmgvvmF2QY/VsrGKAEv_nI/AAAAAAAASxI/jLFpJiCg3RY/s400/7.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="text-align: left;">點選「Create device type」的選項。</span></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-2sdAxmdXmk4/VsrGufRfC7I/AAAAAAAASxQ/LCYDRz3S2dI/s1600/8.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="191" src="https://3.bp.blogspot.com/-2sdAxmdXmk4/VsrGufRfC7I/AAAAAAAASxQ/LCYDRz3S2dI/s400/8.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
輸入裝置的一般資訊,包含名稱、描述,筆者這邊分別填「Vehicle」與「Internet of things」,然後點選Next。</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-WsAumyFQt10/VsrHjOzwlKI/AAAAAAAASxc/Yk-dnzGKws8/s1600/9.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="221" src="https://4.bp.blogspot.com/-WsAumyFQt10/VsrHjOzwlKI/AAAAAAAASxc/Yk-dnzGKws8/s400/9.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
以下的幾個步驟,都是針對裝置的細部描述做紀錄,建議都先以「Next」做跳過的動作,</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-WieE-q0C9xk/VsrH8ONusCI/AAAAAAAASxk/UogcknEqzRI/s1600/10.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="231" src="https://1.bp.blogspot.com/-WieE-q0C9xk/VsrH8ONusCI/AAAAAAAASxk/UogcknEqzRI/s400/10.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-vpnzSU7H5T4/VsrH7xnMJlI/AAAAAAAASxg/vD2GI03bpA8/s1600/11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="228" src="https://3.bp.blogspot.com/-vpnzSU7H5T4/VsrH7xnMJlI/AAAAAAAASxg/vD2GI03bpA8/s400/11.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-u2HpW6n1Omo/VsrH8GhDiKI/AAAAAAAASxo/8jfoSlKSJzM/s1600/12.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="226" src="https://2.bp.blogspot.com/-u2HpW6n1Omo/VsrH8GhDiKI/AAAAAAAASxo/8jfoSlKSJzM/s400/12.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
創建完基本的裝置樣本之後,直接在下個畫面中選擇建好的「Vehicle」裝置樣本,並點選「Next」進行裝置的建立。</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-emVMxwgNW0g/VsrKRSSayAI/AAAAAAAASx4/v2voi5JEwT4/s1600/13.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="231" src="https://2.bp.blogspot.com/-emVMxwgNW0g/VsrKRSSayAI/AAAAAAAASx4/v2voi5JEwT4/s400/13.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們新增第一個車聯網裝置「vehicle1」</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-2V9_KgeIWGM/VsrLsF7wA1I/AAAAAAAASyE/5LzWw9KhRBg/s1600/14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="231" src="https://1.bp.blogspot.com/-2V9_KgeIWGM/VsrLsF7wA1I/AAAAAAAASyE/5LzWw9KhRBg/s400/14.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
接著一直下一步到取得裝置的連接號碼:</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-z_M4AyZ8TC0/VsrMP_kW7iI/AAAAAAAASyM/k7_K5vZ6eCw/s1600/15.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://1.bp.blogspot.com/-z_M4AyZ8TC0/VsrMP_kW7iI/AAAAAAAASyM/k7_K5vZ6eCw/s320/15.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-BUf_fF-yPhg/VsrMQIwuUfI/AAAAAAAASyQ/ZLHEuf1uKoQ/s1600/16.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="183" src="https://3.bp.blogspot.com/-BUf_fF-yPhg/VsrMQIwuUfI/AAAAAAAASyQ/ZLHEuf1uKoQ/s320/16.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-H4tHoPU1YCE/VsrMQKXPBNI/AAAAAAAASyU/f2ZVkgchrWo/s1600/17.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="184" src="https://4.bp.blogspot.com/-H4tHoPU1YCE/VsrMQKXPBNI/AAAAAAAASyU/f2ZVkgchrWo/s320/17.png" width="320" /></a></div>
<br />
注意:到這裡的時候,就要記錄下vehicle1的存取ID與相關資料。<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-10-ZvBcoTPI/VsrMQpLwsUI/AAAAAAAASyY/e4hDp3-wG4k/s1600/18.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="440" src="https://3.bp.blogspot.com/-10-ZvBcoTPI/VsrMQpLwsUI/AAAAAAAASyY/e4hDp3-wG4k/s640/18.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
筆者在這個時候就會把上面紅框的資料做紀錄,等下要放到車聯網平台上。</div>
<div class="separator" style="clear: both; text-align: left;">
接著以此類推,做兩次,將「vehicle2」、「vehicle3」完成另外兩個裝置的設定。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著我們到「ACCESS」去獲取組織的ID,如下圖所示:</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://4.bp.blogspot.com/-wQ_zg5_T4kM/VsrNQN4KxBI/AAAAAAAASyk/yHmAVblx4U8/s1600/20.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="175" src="https://4.bp.blogspot.com/-wQ_zg5_T4kM/VsrNQN4KxBI/AAAAAAAASyk/yHmAVblx4U8/s400/20.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
選擇「api keys」然後點選「Generate API Key」,如下圖所示:</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-IPo85_8r5Ac/VsrNdH6pF0I/AAAAAAAASyo/zU2XUwCgx9U/s1600/21.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="150" src="https://1.bp.blogspot.com/-IPo85_8r5Ac/VsrNdH6pF0I/AAAAAAAASyo/zU2XUwCgx9U/s400/21.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
接著紀錄一下下圖中的API Key,等下我們會放到程式當中。</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://3.bp.blogspot.com/-uudOFFgTOMA/VsrN0AwVf6I/AAAAAAAASyw/bADyj83NS7M/s1600/22.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="362" src="https://3.bp.blogspot.com/-uudOFFgTOMA/VsrN0AwVf6I/AAAAAAAASyw/bADyj83NS7M/s640/22.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
到這一步,終於完成IBM Bluemix上的裝置設定,接下來我們要寫一個範例去進行佈署的動作,筆者修改了一個國外的範例給大家<a href="https://github.com/rippleblue/Bluemix/blob/master/connected_car.zip?raw=true" target="_blank">下載</a>,如果無法下載可到筆者<a href="https://github.com/rippleblue/Bluemix/blob/master/connected_car.zip" target="_blank">GitHub下載</a>,裡面包含了兩份檔案:</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ol>
<li>檔案「iotcc」為預計佈署到雲端上的車聯網軟體,主要用Node.js、HTML、CSS所撰寫</li>
<li>檔案「cf-cli-installer_6.15.0_winx64」進行Git Push的工具</li>
</ol>
<div>
首先解壓縮後打開「manifest.yml」檔案,修改您的服務名稱,以下圖為例:</div>
<div>
請輸入</div>
<div>
<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="b41824ae-9962-4e1d-a44b-d8d0fcc424a6" id="64f04e15-2de7-4289-b93f-1d361777e810"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="69db9c82-09c7-4dcf-80ee-15e740a9193a" id="f165e4a1-00e0-4caa-8d4c-842eba9a8c8c"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="c7f261ef-ecaa-4232-8f17-dd6399869c1a" id="03dbed8e-d6d7-4fc6-a55d-78325785dc12">host</gs></gs></gs><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="b41824ae-9962-4e1d-a44b-d8d0fcc424a6" id="9e5afdc8-4c3f-49ce-9cb8-d788e313b495"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="69db9c82-09c7-4dcf-80ee-15e740a9193a" id="a1762170-8828-4a39-94c4-1aa416486598"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="c7f261ef-ecaa-4232-8f17-dd6399869c1a" id="d50b262c-f5dc-4d7a-aedd-64592cca1517">:</gs></gs></gs><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="b41824ae-9962-4e1d-a44b-d8d0fcc424a6" id="2b5cc427-0c74-4236-ba5a-711f6322c81d"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="69db9c82-09c7-4dcf-80ee-15e740a9193a" id="5e8bec79-daee-4db9-bab0-f91aaef06017"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="c7f261ef-ecaa-4232-8f17-dd6399869c1a" id="ce60d053-40eb-4d1a-b6fd-29d380ba6e11">connectedcar</gs></gs></gs></div>
<div>
<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="78dc4d26-125b-403f-92ff-40440ed9009e" id="c2983c38-521c-4c28-b3de-990ffe944079"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="8e34e63b-e732-4e89-935e-46ff5e00c12b" id="b7259916-0aa9-4e77-9ce1-6341d1026e3f"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="bddbb45e-0ef5-4a97-b912-71ab1969296c" id="05def820-9dd5-450d-aeb0-b93d29cc75b1">name</gs></gs></gs><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="78dc4d26-125b-403f-92ff-40440ed9009e" id="10d3ec86-eea3-4fd4-bd74-1afb0c7ac452"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="8e34e63b-e732-4e89-935e-46ff5e00c12b" id="1aaaaa23-4f70-487f-8073-740bf12b61af"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="bddbb45e-0ef5-4a97-b912-71ab1969296c" id="ea0c2ca1-5a50-4667-b2b4-c3168500f074">:</gs></gs></gs><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="78dc4d26-125b-403f-92ff-40440ed9009e" id="17b1fc5e-a3e5-4558-949f-8c86bd6dcdec"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="8e34e63b-e732-4e89-935e-46ff5e00c12b" id="bd357c65-82c7-4bb9-913d-1b043eb65971"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="bddbb45e-0ef5-4a97-b912-71ab1969296c" id="600b2804-8cbb-4d15-9cf8-1dd088f575a9">connectedcar</gs></gs></gs></div>
<div>
<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="7163b985-1229-443b-a0ee-4a0c89edb196" id="0bb119b8-5ca3-4158-acda-fbae3693b59f"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="78fa2e27-3354-47c6-9e85-e9b77981346c" id="d27959cb-b6c9-4258-8aeb-8e9c1dc50a17"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="e0f54ceb-ac7b-42f9-a7b2-bb258e8fd4c8" id="bce9460c-1cf8-468e-8589-1521a848c53a">instances</gs></gs></gs><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="7163b985-1229-443b-a0ee-4a0c89edb196" id="6a33a441-5d73-44d0-a4ce-905f786ad50b"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="78fa2e27-3354-47c6-9e85-e9b77981346c" id="ff8bc84e-c00a-4ec0-96b0-2e266510c609"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="e0f54ceb-ac7b-42f9-a7b2-bb258e8fd4c8" id="576d459c-576e-4475-b72e-4f8e0ee22408">:</gs></gs></gs>3 (指3台裝置)</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-DIi8V-PXy-M/VsrSqEbel-I/AAAAAAAASzE/-fLgoIqMMDA/s1600/23.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="https://1.bp.blogspot.com/-DIi8V-PXy-M/VsrSqEbel-I/AAAAAAAASzE/-fLgoIqMMDA/s320/23.png" width="320" /></a></div>
<div>
接著到public\config資料夾裡面,打開「settings.js」檔案,將前面申請的裝置與組織ID都輸入到該檔案當中,如下:</div>
<div>
iot_deviceType<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="a668c0f3-0efe-40f3-b099-fce9dd315232" id="0d38c554-0c79-4766-8ad4-4cf37ba3b113"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="c6628992-dc7f-42aa-8a9f-8f7cc7128939" id="32f8ba10-9ad7-47da-8b66-26fb5de56204"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="6c316c94-bd34-41a1-95d3-1a230776fd29" id="e2a5c9ef-52a7-4e94-8bad-4244b02da9a0">:</gs></gs></gs></div>
<div>
輸入我們在前面建立的「Vehicle」。</div>
<div>
iot_deviceOrg<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="6f125c48-29a8-44ac-b3f6-082385a6907c" id="48352187-40da-4099-a9af-1432bb78c9c3"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="864d7e2e-4b1b-43f1-81fc-342db70a09bf" id="911305a9-32c2-4577-ac7f-c6a1dc00b426"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="80b49805-2b5c-41b1-bb1a-f9c47bd731c7" id="2d28500a-1798-4053-b024-c7ff560c2084">:</gs></gs></gs></div>
<div>
輸入在前面裝置申請時獲得的「Organization ID」,如:k8buaf。</div>
<div>
iot_deviceSet中的deviceId與token:</div>
<div>
輸入前面裝置申請獲得的Device ID<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="ca2dc206-7f8f-4ef7-9afc-0856948f999e" id="73c72031-2276-48ae-97d8-62b27bf67427">,</gs>token的部分則輸入Authentication Token如:A!ggkk99y_sstbt)vY</div>
<div>
<br /></div>
<div>
最後是填入組織的ID</div>
<div>
<div>
iot_apiKey<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="2922145e-a971-404d-ab0a-37b7136ee895" id="aa54b54b-3606-468b-87d3-1d3314e344a9"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="3abe882c-c76b-4af8-b512-454b0c9f4471" id="b3540955-947c-4020-a674-b5d72fccf4a6"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="d5a1d1e6-0b95-4c4a-bedf-88e13a8b1d82" id="10754989-0083-40f6-a978-864db705c8d1">:</gs></gs></gs>輸入前面申請到的API Key</div>
<div>
iot_apiToken<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="d79a4c40-f08c-48a5-a85d-0fc59ae678f7" id="eb062e08-3ee5-424f-9ff3-debda2619c55"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="d53de492-3dac-4083-8134-9d0778551014" id="e7723530-fb75-42eb-b5d5-b8478888f4bc"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="965ea19e-5135-445e-8ca3-0043f0a81d13" id="3b81d90b-5d6b-43e6-9a9d-2dda7b9251c5">:</gs></gs></gs>輸入前面申請的Authentication Toke</div>
</div>
<div>
<br /></div>
<div>
大致上設定完成,最後我們要Cloud Foundry的工具,將車連網服務佈署到雲端平台上,因此要安裝「cf_installer」,基本上一直下一步就可完成安裝。</div>
<div>
<br /></div>
<div>
接著我們打開CMD,進行登入的動作,如下圖所示,筆者先切換到iotcc資料夾底下,打入「cf login」的動作,使用一開始註冊的帳號密碼登入即可。</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-fwmghQ-SbMc/VsrXHiGAfII/AAAAAAAASzQ/0otr6Y6JzPs/s1600/25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="118" src="https://1.bp.blogspot.com/-fwmghQ-SbMc/VsrXHiGAfII/AAAAAAAASzQ/0otr6Y6JzPs/s400/25.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
登入成功後會顯示下面畫面。</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-EOqUw_p7bR4/VsrXdFoVEZI/AAAAAAAASzU/Fn9XFf6jp5E/s1600/26.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-EOqUw_p7bR4/VsrXdFoVEZI/AAAAAAAASzU/Fn9XFf6jp5E/s1600/26.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
最後打入「cf push」進行Deploy的動作</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-ZfIkYH1KeRU/VsrXjhbyrYI/AAAAAAAASzY/_qVy7l-iQe4/s1600/27.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="76" src="https://1.bp.blogspot.com/-ZfIkYH1KeRU/VsrXjhbyrYI/AAAAAAAASzY/_qVy7l-iQe4/s400/27.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
看到下圖畫面表示已經佈署成功。</div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://1.bp.blogspot.com/-IiyRboMv3Ls/VsrX4bwE8BI/AAAAAAAASzg/KZlu0ZFMAO0/s1600/28.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="267" src="https://1.bp.blogspot.com/-IiyRboMv3Ls/VsrX4bwE8BI/AAAAAAAASzg/KZlu0ZFMAO0/s640/28.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
最後輸入http://<您設定的host名稱>.mybluemix.net,即可看到服務,也可參考筆者下圖成果。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://2.bp.blogspot.com/-FtUl1-5MvAA/VsrYsmcLIuI/AAAAAAAASzw/OH32npEDrDY/s1600/22.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="442" src="https://2.bp.blogspot.com/-FtUl1-5MvAA/VsrYsmcLIuI/AAAAAAAASzw/OH32npEDrDY/s640/22.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
接著可以透過「互動」的功能達到跟模擬汽車互動的動作。</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
時間的關係,這次就先暫時介紹到這邊吧!</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0tag:blogger.com,1999:blog-5898372732399324805.post-88457958665685898262016-01-26T14:50:00.000+08:002017-03-05T17:16:14.212+08:002016年台灣巨量資料(大數據)、資料科學家相關課程 (更新時間2016.07.17)<br />
很快的新的一年又開始了,筆者已經持續整理資料科學相關課程長達兩年,今年也一樣整理相關課程與上課時間給有需要的朋友查詢。<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-XXjdqL18jKk/WLvXNxy7c-I/AAAAAAAATbw/zrVxC9DzcoQLuq4qvUM_QIaVyFI1hSyfACEw/s1600/courses-icon-6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="200" src="https://2.bp.blogspot.com/-XXjdqL18jKk/WLvXNxy7c-I/AAAAAAAATbw/zrVxC9DzcoQLuq4qvUM_QIaVyFI1hSyfACEw/s200/courses-icon-6.png" width="200" /></a></div>
<br />
<br />
工研院<br />
<ul>
<li><a href="https://college.itri.org.tw/SeminarView.aspx?no=23160196&msgno=314700" target="_blank">資料探勘與R軟體應用實務</a>(2016/02/23~ 02/24)</li>
<li><a href="https://college.itri.org.tw/SeminarView.aspx?no=23160157&msgno=314674" target="_blank">巨量資料分析應用與實作班</a>(2016/03/09、03/16、03/23)</li>
<li><a href="https://college.itri.org.tw/SeminarView.aspx?no=23160213&msgno=314714" target="_blank">資料探勘與文本數據實作</a>(2016/3/14、3/15)</li>
<li><a href="https://college.itri.org.tw/SeminarView.aspx?no=23160191&msgno=314697" target="_blank">巨量資料分析智慧製造應用~設備故障預診斷與健康管理系統</a>(2016/05/26、05/27)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=6F5837B2-4266-4267-93F4-AD91DC071BFB" target="_blank">大數據BigData於Excel資料智慧分析應用培訓班</a>(2016/08/28~09/11)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=002DA6B5-8F0D-4D2A-B6B3-BA70938F23F0" target="_blank">使用專利大數據分析建立專利地圖之實務研習班</a>(2016/08/16~08/17)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=2B85FC4B-FAFD-42EF-BF81-18BD8D579579" target="_blank">雲端ERP系統結合物聯網及大數據之應用</a>(2016/10/05~10/07)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=011B25DB-6C14-4BFB-9274-DAC21F7327F2" target="_blank">工業4.0的核心-大數據與小數據的分析</a>(2016/10/25~10/25)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=85F8036F-1B41-4939-971A-95C596805735" target="_blank">物聯網時代的大數據智能應用專題演講</a>(2016/08/22~08/22)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=3523F8A1-84BB-4FC7-ADD0-E255C6E6704F" target="_blank">巨量資料分析智慧製造應用~設備故障預診斷與健康管理系統</a>(2016/09/29~09/30)</li>
<li><a href="https://college.itri.org.tw/SeminarView2.aspx?posno=97EAD4C8-20AC-408A-A357-00910C9A3179" target="_blank">巨量資料分析應用與實作班</a>(2016/08/09~08/23)</li>
</ul>
<div>
資策會</div>
<div>
<ul>
<li><a href="http://www.iiiedu.org.tw/ites/RD.htm" target="_blank">Big Data資料分析二部曲-R軟體與資料探勘</a>(2016/1/30~1/31)</li>
<li><a href="http://www.iiiedu.org.tw/ites/TEXT.htm" target="_blank">文字資料探勘實作班</a>(2016/2/20、2/21)</li>
<li><a href="http://www.iiiedu.org.tw/ites/CBDA.htm" target="_blank">Big Data之處理與分析<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="0644e1dd-5ae4-46fb-bdbb-02abc1a48400" id="3a52444a-78ec-48bb-b69e-2511d8b75224"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="b9bda3e6-e0ba-49c6-8efb-558d916e3f02" id="98f123f6-ca60-45c3-852f-df1fcf5cc7f3"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="c34a4e06-f0f0-4662-85de-d733d2065a0c" id="5f6fc535-1aac-4a05-ba8a-33ef2e538daa">(</gs></gs></gs>Hadoop<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="0644e1dd-5ae4-46fb-bdbb-02abc1a48400" id="6ba074fb-6ec2-4b82-af21-63f9d2d88bcd"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="b9bda3e6-e0ba-49c6-8efb-558d916e3f02" id="5a8e47f0-df79-43cf-9002-2a9ea5e038fc"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="c34a4e06-f0f0-4662-85de-d733d2065a0c" id="b76b42a0-034e-4ea9-8c60-1fe14499ee99">)</gs></gs></gs>進階班</a><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="0644e1dd-5ae4-46fb-bdbb-02abc1a48400" id="c7eefeaa-1ad4-4bca-ab89-c26dc9738293"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="b9bda3e6-e0ba-49c6-8efb-558d916e3f02" id="ab64be32-8806-43d3-b0df-4d193e07d886"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="c34a4e06-f0f0-4662-85de-d733d2065a0c" id="8f6f8fcb-ac1a-4339-8c45-ae19dc0526aa">(</gs></gs></gs>2016/2/20~2/21)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RB.htm" target="_blank">Big Data資料分析首部曲-R軟體實作</a>(2016/3/5、3/6)</li>
<li><a href="http://www.iiiedu.org.tw/ites/PYTHON.htm" target="_blank">Python資料探勘實作</a>(2016/3/12、3/13)</li>
<li><a href="http://www.iiiedu.org.tw/ites/CBD.htm" target="_blank">Big Data之處理與分析<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="01d179b8-e47a-495f-b635-1f4487e8a773" id="43fb28fe-413d-4d2d-96a7-1a5df442021f"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="233a1387-41d4-44f0-9451-5a7d263deb03" id="4e72c61c-b738-428e-8d86-312ff664590f"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="72973695-4ba8-4840-a9b5-1e4d32db2682" id="d2c3768b-880d-43e7-a663-e54cf80c7411">(</gs></gs></gs>Hadoop<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="01d179b8-e47a-495f-b635-1f4487e8a773" id="fede35d0-768e-403f-97b0-46f3cde5f4a7"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="233a1387-41d4-44f0-9451-5a7d263deb03" id="8e035d03-5e4f-450a-b2f6-630b12774140"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="72973695-4ba8-4840-a9b5-1e4d32db2682" id="e3154eb3-2935-46a3-97e3-7a655cd89d52">)</gs></gs></gs>實務班</a><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="01d179b8-e47a-495f-b635-1f4487e8a773" id="bf6d7192-a798-4bc6-bb0d-a7422f5d63ce"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="233a1387-41d4-44f0-9451-5a7d263deb03" id="69432801-07aa-4dc9-807f-f1c4f4913e7b"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="72973695-4ba8-4840-a9b5-1e4d32db2682" id="95490b01-70e4-4015-8253-c8d5835d54f0">(</gs></gs></gs>2016/3/12~3/13)</li>
<li><a href="http://www.iiiedu.org.tw/ites/ELAS.htm" target="_blank">Elasticsearch分散式系統實務班</a>(2016/3/19、3/20)</li>
<li><a href="http://www.iiiedu.org.tw/ites/SPARK.htm" target="_blank">Spark巨量資料分析實務班</a>(2016/3/26、3/27) </li>
<li><a href="http://www.iiiedu.org.tw/ites/VRD.htm" target="_blank">視覺式資料探勘實作</a>(2016/3/26、3/27)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RV.htm" target="_blank">Big Data資料分析三部曲-R軟體與資料視覺化</a>(2016/3/26~3/27)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RFIN.htm" target="_blank">R軟體與財金資料分析</a>(2016/4/9、4/10)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RWP.htm" target="_blank">R軟體與網頁資料擷取應用</a>(2016/4/16、4/17)</li>
<li><a href="http://www.iiiedu.org.tw/ites/SST.htm" target="_blank">Spark Streaming巨量資料串流分析實務班</a><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="f22a90d1-dce6-40d1-9686-6431527b16b4" id="a1d91797-4faf-4435-8bdd-3a4e0b980348"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="72171276-b97f-4804-a874-a5ab45e16539" id="fdfc92da-25a8-45cc-9ae5-8d7315fee862"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="7b6219be-b151-4797-bfba-9e71c6463aaf" id="c05d5450-2b5c-4011-89cf-7cf55104da2d">(</gs></gs></gs>2016/4/23<gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="f22a90d1-dce6-40d1-9686-6431527b16b4" id="9b9189f6-d33a-4c83-a48b-58edc37f42a6"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="72171276-b97f-4804-a874-a5ab45e16539" id="5467abf0-2af8-4647-bd19-28b25da751a0"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="7b6219be-b151-4797-bfba-9e71c6463aaf" id="7080238b-c29f-43d1-9e3f-7a89b2182658">、</gs></gs></gs>4/24)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RHADOOP.htm" target="_blank">巨量資料分析-使用RHadoop</a></li>
<li><a href="http://www.iiiedu.org.tw/ites/TEXT.htm" target="_blank">文字資料探勘實作班</a>(2016/7/23、7/24)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RV.htm" target="_blank">Big Data資料分析三部曲-R軟體與資料視覺化</a>(2016/ 7/23~7/24)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RWP.htm" target="_blank">R軟體與網頁資料擷取應用</a>(2016/7/30、7/31)</li>
<li><a href="http://www.iiiedu.org.tw/ites/CBDA.htm" target="_blank">Big Data之處理與分析(Hadoop)進階班</a>(2016/8/6、8/7)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RFIN.htm" target="_blank">R軟體與財金資料分析</a>(2016/8/13、8/14)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RST.htm" target="_blank">R軟體與串流資料分析</a>(2016/8/20、8/21)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RD.htm" target="_blank">Big Data資料分析首部曲-R軟體實作</a>(2016/8/27、8/28) </li>
<li><a href="http://www.iiiedu.org.tw/ites/PYTHON.htm" target="_blank">Python資料探勘實作</a>(2016/9/3、9/4)</li>
<li><a href="http://www.iiiedu.org.tw/ites/ELAS.htm" target="_blank">Elasticsearch分散式系統實務班</a>(2016/9/3、9/4)</li>
<li><a href="http://www.iiiedu.org.tw/ites/VRD.htm" target="_blank">視覺式資料探勘實作</a>(2016/9/24、9/25)</li>
<li><a href="http://www.iiiedu.org.tw/ites/SPARK.htm" target="_blank">Spark巨量資料分析實務班</a>(2016/9/24、9/25)</li>
<li><a href="http://www.iiiedu.org.tw/ites/RD.htm" target="_blank">Big Data資料分析二部曲-R軟體與資料探勘(</a>2016/10/15、10/16)</li>
<li><a href="http://www.iiiedu.org.tw/ites/SPARKR.htm" target="_blank">巨量資料分析-使用SparkR</a> (2016/10/15、10/16)</li>
<li><a href="http://www.iiiedu.org.tw/ites/CBD.htm" target="_blank">Big Data之處理與分析(Hadoop)實務班</a>(2016/10/29、10/30)</li>
</ul>
<div>
MIC產業顧問學院</div>
</div>
<div>
<ul>
<li><a href="http://mic.iii.org.tw/Institute/course/recent_ins.aspx?sqno=1559" target="_blank">大數據分析與數位行銷技能精修班</a>(2016/3/15~2016/3/29)</li>
</ul>
SAS</div>
<div>
<ul>
<li><a href="http://www.sas.com/apps/wtraining2/trainsched.jsp?ctry=tw" target="_blank">SAS教育訓練</a></li>
</ul>
</div>
三星課程網<br />
<div>
<ul>
<li><a href="http://www.tutortristar.com/topic/Data-mining/seminar-data&bigdata-201601.html" target="_blank">資料治理與大數據分析</a>(2016/1/17)</li>
<li><a href="http://www.tutortristar.com/topic/Data-mining/Data-mining-chen.html" target="_blank">Big Data大數據採礦分析R語言教學</a><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="c8abe2e1-b578-4ff4-931b-13998c5fedc7" id="53d72947-173d-4bdb-8878-f0d623d3b9b9"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="fcb0f2a5-ca05-46af-9c52-21205e975270" id="875ab151-b6b5-42e2-844b-35c11d673f60"><gs class="GINGER_SOFTWARE_mark" ginger_software_uiphraseguid="79f4da0d-3daf-4742-b41c-d3821404ec00" id="e0f8c803-7234-48e7-84ff-600bbafaad05">(</gs></gs></gs>2016/1/23-1/24)</li>
</ul>
</div>
<div>
木刻思</div>
<div>
<ul>
<li><a href="http://datasci.kktix.cc/events/scu-datacamp-2016winter" target="_blank">2016「資料科學首部曲-數據甦醒」</a>(2016/01/27~01/28)</li>
</ul>
</div>
<div>
東吳大學</div>
<div>
<ul>
<li><a href="http://www.ext.scu.edu.tw/class_0a.aspx?ClassID=1940&ClassTypeID=3&SubClassID=31" target="_blank">大數據分析基礎與應用-挖資料的金礦</a>(2016/3/2~4/27)</li>
</ul>
<div>
文化大學</div>
</div>
<div>
<ul>
<li><a href="http://www.1111edu.com.tw/advancedStudies_courseContent.php?autono=125678" target="_blank">大數據行銷</a>(2016/01/29)</li>
</ul>
<div>
<br />
<br />
<br /></div>
</div>
<div>
歷年資料參考:</div>
<div>
<a href="http://dataology.blogspot.tw/2015/05/2015-201505.html" target="_blank">2015年台灣巨量資料(大數據)、資料科學家相關課程</a></div>
<div>
<a href="http://dataology.blogspot.tw/2014/01/2014big-data.html" target="_blank">2014年台灣巨量資料、資料科學家相關課程</a></div>
<div>
<br /></div>
<div>
<br /></div>
資料科學實驗室http://www.blogger.com/profile/08773841779420148582noreply@blogger.com0