| 摘要: |
| 地面自动站气象数据的量级、观测传输频次日益增长,传输方式从传统的文本方式发展到消息传输,对数据处理时效有了更高的要求。本文提出采用Storm流式技术进行地面自动站气象数据的解码和入库,使用Spout节点连接外部数据源,将数据转化为Tuple,传递给解码Bolt、入库Bolt和DI/EI消息Bolt,分别进行实时的入库和监控,且达到灵活调整系统资源的目的。该应用与传统的简约流程对比,稳定性、处理效率、拓展性都有了显著提高。 |
| 关键词: 地面气象数据; 海量数据; Storm流式技术; 高效处理。 |
| DOI： |
| 投稿时间：2019-04-23修订日期：2019-08-05 |
| 基金项目:贵州省科技计划项目([2019]2386): 基于大数据的气象防灾减灾服务平台研发 |
|
| Application of Storm Flow Technology in Surface Meteorological Data Processing |
| Liao Tingting,Wang Biao,Xiao Weiqing,Li Congying,Guo Xi |
| (Guizhou Meteorological Information Center,Guiyang;National Meteorological Information Center,Beijing) |
| Abstract: |
| The quantity and observational frequency of meteorological data by surficial automatic station are increasing. The transmission mode has developed from the traditional text mode to the message transmission mode. These changes make higher requirement for data processing timeliness. In this paper, we used Storm Flow Technology on decoding and warehouse of surficial automatic station data. It used Spout to connect external data sources, and convert data into Tuple, and then transmit to Bolt-decode, Bolt-warehouse and Bolt-DIEI. They can be used to real-time warehouse, monitoring, and quickly adjust system resource. Compared with the traditional simplified process, the stability, processing efficiency and expansibility of the application have been greatly improved. |
| Key words: automaic weather station data; big data; storm flow technology; high-activity processing. |
