成果報告書詳細
管理番号20130000000495
タイトル*平成24年度中間年報 IT融合による新社会システムの開発・実証プロジェクト (データ処理基盤分野)リアルタイム大規模データ解析処理基盤の研究開発
公開日2013/10/1
報告書年度2012 - 2012
委託先名日本電気株式会社 株式会社Preferred Infrastructure 独立行政法人産業技術総合研究所
プロジェクト番号P12009
部署名電子・材料・ナノテクノロジー部
和文要約
英文要約Title: IT Integration-based New Social System Development and Demonstration Projects / Data processing infrastructure field / Development of Scalable Real-time Processing Platforms for Large-scale Data Analysis

(FY2012-2013) FY2012 Annual Report

[1. Research and development of real-time data analysis technologies]
1-1 Real-time analysis of large-scale time series data:
We designed and implemented a performance analysis platform for Jubatus. The platform enables to compare multiple methods including existing systems such as Mahout on Hadoop, and several versions of Jubatus. We also researched fundamental and existing technologies and real applications and confirmed a feasibility of realizing targets in next year, in terms of algorithm refinement, multi-core optimization and performance refinement of feature extraction.
1-2 data analysis technology with a wide variety of information: We designed and implemented a prototype feature extraction module by integrating CHLAC and OpenCV into Jubatus, and confirmed a feasibility of performance goal to process eight 20-FPS video streams in parallel. We also confirmed a feasibility of interface enhancement by offloading partial function of the prototype to an external RPC server.
[2. Research and development of next-generation data-store technology]
2-1 High-performance parallel data-store supporting large-scale real-time analysis: We designed semi-real time access treatment technologies for read and write accesses individually to achieve high throughput performance. We also developed a synchronous algorithm among storage nodes by referring specifications of real-time analysis engines and data center network such as utilization pattern of historical data, cross-stream access, and influence of network delay.
2-2 Next-generation memory storage node technology:
We designed a fundamental data management method on PCIe-SSD to enable accesses by different access size between read and write request and searches with timestamps. We also confirmed a feasibility of realizing performance goal and data compression at real applications in terms of input data processing accelerator by integrating novel cache structure. The cache enables to guarantee read performance even if the real application causes bursty write workload.
2-3 Next-generation memory storage device technology:
We firstly evaluated devices including flash-memory, DRAM, memory controller, and models the devices with Systems. Second, we integrated the models into a hardware simulator with storage engine middleware to evaluate the storage devices.

ダウンロード成果報告書データベース(ユーザ登録必須)から、ダウンロードしてください。

▲トップに戻る