Welcome to the HyperFORGE

The HyperFORGE was created to facilitate collaboration on plugins and documentation for they Hyperic Community.

Become a HyperFORGE member →

Recent Activity

Skip to end of metadata
Go to start of metadata

Hadoop plugin for HQ

Several folks have been asking about a Hadoop plugin for HQ... we've posted an early version of plugin available at: http://github.com/hyperic/hq-hadoop

Feedback and contributions are welcome of course, fork away!

Generated plugin metrics follow...

Hadoop Data Node server

Log Track Supported: Yes
Config Track Supported: Yes
Custom Properties Supported:

Name Description
VmVersion Vm Version
VmVendor Vm Vendor
VmName Vm Name

Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Process Virtual Memory Size ProcessVirtualMemorySize B UTILIZATION false 5 min
Process Resident Memory Size ProcessResidentMemorySize B UTILIZATION true 5 min
Process Page Faults ProcessPageFaults none UTILIZATION false 10 min
Process Page Faults per Minute ProcessPageFaults1m none UTILIZATION false 10 min
Process Cpu System Time ProcessCpuSystemTime ms UTILIZATION false 10 min
Process Cpu System Time per Minute ProcessCpuSystemTime1m ms UTILIZATION false 10 min
Process Cpu User Time ProcessCpuUserTime ms UTILIZATION false 10 min
Process Cpu User Time per Minute ProcessCpuUserTime1m ms UTILIZATION false 10 min
Process Cpu Total Time ProcessCpuTotalTime ms UTILIZATION false 10 min
Process Cpu Total Time per Minute ProcessCpuTotalTime1m ms UTILIZATION false 10 min
Process Cpu Usage ProcessCpuUsage percentage UTILIZATION true 5 min
Process Start Time ProcessStartTime epoch-millis AVAILABILITY false 5 min
Process Open File Descriptors ProcessOpenFileDescriptors none UTILIZATION false 5 min
Process Threads ProcessThreads none UTILIZATION false 5 min
Availability Availability percentage AVAILABILITY true 5 min
Heap Memory Free HeapMemoryFree B UTILIZATION true 5 min
Thread Count ThreadCount none UTILIZATION true 5 min
Total HotSpot Compilation Time TotalHotSpotCompilationTime ms PERFORMANCE true 5 min

Configuration help:

General Log and Config Track Properties

  • Enable Config Tracking - Check to enable config tracking.
  • Configuration Files - Comma delimited list of configuration files to track. Relative files are resolved to ${installpath}.
  • Enable Log Tracking - Check to enable log tracking.
  • Track event log level - Only track events of level greater than or equal to this level. Order is: [Error, Warn, Info, Debug]
    Mapping:
    • FATAL,ERROR -> Error
    • WARN -> Warn
    • INFO -> Info
    • DEBUG -> Debug
  • Log Pattern Match - Include messages that match the given regular expression. The given pattern can be a substring to look for in log messages or a regular expression. See: java.util.regex.Pattern.
  • Log Pattern Exclude - Exclude messages that match the given regular expression.
  • Log Files - Comma delimited list of log files to track. Relative files are resolved to ${installpath}.

Hadoop Data Node FS Dataset service

Log Track Supported: No
Config Track Supported: No
Custom Properties Supported: None
Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Availability Availability percentage AVAILABILITY true 10 min
Remaining Remaining B UTILIZATION true 5 min
Capacity Capacity B UTILIZATION true 5 min
DfsUsed DfsUsed B UTILIZATION true 5 min

Configuration help:

Hadoop Data Node RPC service

Log Track Supported: No
Config Track Supported: No
Custom Properties Supported: None
Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Availability Availability percentage AVAILABILITY true 10 min
Num Open Connections NumOpenConnections none UTILIZATION true 5 min
Call Queue Len CallQueueLen none UTILIZATION false 5 min
Rpc Ops Avg Processing Time RpcOpsAvgProcessingTime ms UTILIZATION false 5 min
Rpc Ops Avg Processing Time Max RpcOpsAvgProcessingTimeMax ms UTILIZATION false 5 min
Rpc Ops Avg Processing Time Min RpcOpsAvgProcessingTimeMin ms UTILIZATION false 5 min
Rpc Ops Avg Queue Time RpcOpsAvgQueueTime ms UTILIZATION true 5 min
Rpc Ops Avg Queue Time Max RpcOpsAvgQueueTimeMax ms UTILIZATION false 5 min
Rpc Ops Avg Queue Time Min RpcOpsAvgQueueTimeMin ms UTILIZATION false 5 min
Rpc Ops Number RpcOpsNumber none UTILIZATION false 5 min

Configuration help:

Hadoop Data Node Stats service

Log Track Supported: No
Config Track Supported: No
Custom Properties Supported: None
Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Availability Availability percentage AVAILABILITY true 10 min
Bytes Read BytesRead B UTILIZATION false 10 min
Bytes Read per Minute BytesRead1m B UTILIZATION true 10 min
Bytes Written BytesWritten B UTILIZATION false 10 min
Bytes Written per Minute BytesWritten1m B UTILIZATION true 10 min
Blocks Read BlocksRead none UTILIZATION false 10 min
Blocks Read per Minute BlocksRead1m none UTILIZATION false 10 min
Blocks Removed BlocksRemoved none UTILIZATION false 10 min
Blocks Removed per Minute BlocksRemoved1m none UTILIZATION false 10 min
Blocks Replicated BlocksReplicated none UTILIZATION false 10 min
Blocks Replicated per Minute BlocksReplicated1m none UTILIZATION false 10 min
Blocks Written BlocksWritten none UTILIZATION false 10 min
Blocks Written per Minute BlocksWritten1m none UTILIZATION false 10 min
Block Verification Failures BlockVerificationFailures none UTILIZATION false 10 min
Block Verification Failures per Minute BlockVerificationFailures1m none UTILIZATION false 10 min
Blocks Verified BlocksVerified none UTILIZATION false 10 min
Blocks Verified per Minute BlocksVerified1m none UTILIZATION false 10 min
Reads From Local Client ReadsFromLocalClient none UTILIZATION false 10 min
Reads From Local Client per Minute ReadsFromLocalClient1m none UTILIZATION false 10 min
Reads From Remote Client ReadsFromRemoteClient none UTILIZATION false 10 min
Reads From Remote Client per Minute ReadsFromRemoteClient1m none UTILIZATION false 10 min
Writes From Local Client WritesFromLocalClient none UTILIZATION false 10 min
Writes From Local Client per Minute WritesFromLocalClient1m none UTILIZATION false 10 min
Writes From Remote Client WritesFromRemoteClient none UTILIZATION false 10 min
Writes From Remote Client per Minute WritesFromRemoteClient1m none UTILIZATION false 10 min
Read Block Op Average Time ReadBlockOpAverageTime none UTILIZATION false 5 min
Read Block Op Max Time ReadBlockOpMaxTime none UTILIZATION false 5 min
Read Block Op Min Time ReadBlockOpMinTime none UTILIZATION false 5 min
Read Block Op Num ReadBlockOpNum none UTILIZATION false 5 min
Read Metadata Op Average Time ReadMetadataOpAverageTime none UTILIZATION false 5 min
Read Metadata Op Max Time ReadMetadataOpMaxTime none UTILIZATION false 5 min
Read Metadata Op Min Time ReadMetadataOpMinTime none UTILIZATION false 5 min
Read Metadata Op Num ReadMetadataOpNum none UTILIZATION false 5 min
Block Checksum Op Average Time BlockChecksumOpAverageTime none UTILIZATION false 5 min
Block Checksum Op Max Time BlockChecksumOpMaxTime none UTILIZATION false 5 min
Block Checksum Op Min Time BlockChecksumOpMinTime none UTILIZATION false 5 min
Block Checksum Op Num BlockChecksumOpNum none UTILIZATION false 5 min
Replace Block Op Average Time ReplaceBlockOpAverageTime none UTILIZATION false 5 min
Replace Block Op Max Time ReplaceBlockOpMaxTime none UTILIZATION false 5 min
Replace Block Op Min Time ReplaceBlockOpMinTime none UTILIZATION false 5 min
Replace Block Op Num ReplaceBlockOpNum none UTILIZATION false 5 min
Write Block Op Average Time WriteBlockOpAverageTime none UTILIZATION false 5 min
Write Block Op Max Time WriteBlockOpMaxTime none UTILIZATION false 5 min
Write Block Op Min Time WriteBlockOpMinTime none UTILIZATION false 5 min
Write Block Op Num WriteBlockOpNum none UTILIZATION false 5 min
Copy Block Op Average Time CopyBlockOpAverageTime none UTILIZATION false 5 min
Copy Block Op Max Time CopyBlockOpMaxTime none UTILIZATION false 5 min
Copy Block Op Min Time CopyBlockOpMinTime none UTILIZATION false 5 min
Copy Block Op Num CopyBlockOpNum none UTILIZATION false 5 min
Block Reports Average Time BlockReportsAverageTime none UTILIZATION false 5 min
Block Reports Max Time BlockReportsMaxTime none UTILIZATION false 5 min
Block Reports Min Time BlockReportsMinTime none UTILIZATION false 5 min
Block Reports Num BlockReportsNum none UTILIZATION false 5 min
Heartbeats Average Time HeartbeatsAverageTime none UTILIZATION false 5 min
Heartbeats Max Time HeartbeatsMaxTime none UTILIZATION false 5 min
Heartbeats Min Time HeartbeatsMinTime none UTILIZATION false 5 min
Heartbeats Num HeartbeatsNum none UTILIZATION false 5 min

Configuration help:

Hadoop Job Tracker server

Log Track Supported: Yes
Config Track Supported: Yes
Custom Properties Supported:

Name Description
VmVersion Vm Version
VmVendor Vm Vendor
VmName Vm Name

Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Process Virtual Memory Size ProcessVirtualMemorySize B UTILIZATION false 5 min
Process Resident Memory Size ProcessResidentMemorySize B UTILIZATION true 5 min
Process Page Faults ProcessPageFaults none UTILIZATION false 10 min
Process Page Faults per Minute ProcessPageFaults1m none UTILIZATION false 10 min
Process Cpu System Time ProcessCpuSystemTime ms UTILIZATION false 10 min
Process Cpu System Time per Minute ProcessCpuSystemTime1m ms UTILIZATION false 10 min
Process Cpu User Time ProcessCpuUserTime ms UTILIZATION false 10 min
Process Cpu User Time per Minute ProcessCpuUserTime1m ms UTILIZATION false 10 min
Process Cpu Total Time ProcessCpuTotalTime ms UTILIZATION false 10 min
Process Cpu Total Time per Minute ProcessCpuTotalTime1m ms UTILIZATION false 10 min
Process Cpu Usage ProcessCpuUsage percentage UTILIZATION true 5 min
Process Start Time ProcessStartTime epoch-millis AVAILABILITY false 5 min
Process Open File Descriptors ProcessOpenFileDescriptors none UTILIZATION false 5 min
Process Threads ProcessThreads none UTILIZATION false 5 min
Availability Availability percentage AVAILABILITY true 5 min
Heap Memory Free HeapMemoryFree B UTILIZATION true 5 min
Thread Count ThreadCount none UTILIZATION true 5 min
Total HotSpot Compilation Time TotalHotSpotCompilationTime ms PERFORMANCE true 5 min

Configuration help:

General Log and Config Track Properties

  • Enable Config Tracking - Check to enable config tracking.
  • Configuration Files - Comma delimited list of configuration files to track. Relative files are resolved to ${installpath}.
  • Enable Log Tracking - Check to enable log tracking.
  • Track event log level - Only track events of level greater than or equal to this level. Order is: [Error, Warn, Info, Debug]
    Mapping:
    • FATAL,ERROR -> Error
    • WARN -> Warn
    • INFO -> Info
    • DEBUG -> Debug
  • Log Pattern Match - Include messages that match the given regular expression. The given pattern can be a substring to look for in log messages or a regular expression. See: java.util.regex.Pattern.
  • Log Pattern Exclude - Exclude messages that match the given regular expression.
  • Log Files - Comma delimited list of log files to track. Relative files are resolved to ${installpath}.

Hadoop Job Tracker RPC service

Log Track Supported: No
Config Track Supported: No
Custom Properties Supported: None
Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Availability Availability percentage AVAILABILITY true 10 min
Num Open Connections NumOpenConnections none UTILIZATION true 5 min
Call Queue Len CallQueueLen none UTILIZATION false 5 min
Rpc Ops Avg Processing Time RpcOpsAvgProcessingTime ms UTILIZATION false 5 min
Rpc Ops Avg Processing Time Max RpcOpsAvgProcessingTimeMax ms UTILIZATION false 5 min
Rpc Ops Avg Processing Time Min RpcOpsAvgProcessingTimeMin ms UTILIZATION false 5 min
Rpc Ops Avg Queue Time RpcOpsAvgQueueTime ms UTILIZATION true 5 min
Rpc Ops Avg Queue Time Max RpcOpsAvgQueueTimeMax ms UTILIZATION false 5 min
Rpc Ops Avg Queue Time Min RpcOpsAvgQueueTimeMin ms UTILIZATION false 5 min
Rpc Ops Number RpcOpsNumber none UTILIZATION false 5 min

Configuration help:

Hadoop Name Node server

Log Track Supported: Yes
Config Track Supported: Yes
Custom Properties Supported:

Name Description
VmVersion Vm Version
VmVendor Vm Vendor
VmName Vm Name

Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Process Virtual Memory Size ProcessVirtualMemorySize B UTILIZATION false 5 min
Process Resident Memory Size ProcessResidentMemorySize B UTILIZATION true 5 min
Process Page Faults ProcessPageFaults none UTILIZATION false 10 min
Process Page Faults per Minute ProcessPageFaults1m none UTILIZATION false 10 min
Process Cpu System Time ProcessCpuSystemTime ms UTILIZATION false 10 min
Process Cpu System Time per Minute ProcessCpuSystemTime1m ms UTILIZATION false 10 min
Process Cpu User Time ProcessCpuUserTime ms UTILIZATION false 10 min
Process Cpu User Time per Minute ProcessCpuUserTime1m ms UTILIZATION false 10 min
Process Cpu Total Time ProcessCpuTotalTime ms UTILIZATION false 10 min
Process Cpu Total Time per Minute ProcessCpuTotalTime1m ms UTILIZATION false 10 min
Process Cpu Usage ProcessCpuUsage percentage UTILIZATION true 5 min
Process Start Time ProcessStartTime epoch-millis AVAILABILITY false 5 min
Process Open File Descriptors ProcessOpenFileDescriptors none UTILIZATION false 5 min
Process Threads ProcessThreads none UTILIZATION false 5 min
Availability Availability percentage AVAILABILITY true 5 min
Heap Memory Free HeapMemoryFree B UTILIZATION true 5 min
Thread Count ThreadCount none UTILIZATION true 5 min
Total HotSpot Compilation Time TotalHotSpotCompilationTime ms PERFORMANCE true 5 min

Configuration help:

General Log and Config Track Properties

  • Enable Config Tracking - Check to enable config tracking.
  • Configuration Files - Comma delimited list of configuration files to track. Relative files are resolved to ${installpath}.
  • Enable Log Tracking - Check to enable log tracking.
  • Track event log level - Only track events of level greater than or equal to this level. Order is: [Error, Warn, Info, Debug]
    Mapping:
    • FATAL,ERROR -> Error
    • WARN -> Warn
    • INFO -> Info
    • DEBUG -> Debug
  • Log Pattern Match - Include messages that match the given regular expression. The given pattern can be a substring to look for in log messages or a regular expression. See: java.util.regex.Pattern.
  • Log Pattern Exclude - Exclude messages that match the given regular expression.
  • Log Files - Comma delimited list of log files to track. Relative files are resolved to ${installpath}.

Hadoop Name Node FS Namesystem service

Log Track Supported: No
Config Track Supported: No
Custom Properties Supported: None
Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Availability Availability percentage AVAILABILITY true 10 min
Capacity Total CapacityTotal B UTILIZATION false 5 min
Capacity Used CapacityUsed B UTILIZATION false 5 min
Capacity Remaining CapacityRemaining B UTILIZATION true 5 min
Total Load TotalLoad none UTILIZATION true 5 min
Blocks Total BlocksTotal none UTILIZATION false 5 min
Files Total FilesTotal none UTILIZATION false 5 min
Pending Replication Blocks PendingReplicationBlocks none UTILIZATION true 5 min
Under Replicated Blocks UnderReplicatedBlocks none UTILIZATION false 5 min
Scheduled Replication Blocks ScheduledReplicationBlocks none UTILIZATION false 5 min

Configuration help:

Hadoop Name Node RPC service

Log Track Supported: No
Config Track Supported: No
Custom Properties Supported: None
Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Availability Availability percentage AVAILABILITY true 10 min
Num Open Connections NumOpenConnections none UTILIZATION true 5 min
Call Queue Len CallQueueLen none UTILIZATION false 5 min
Rpc Ops Avg Processing Time RpcOpsAvgProcessingTime ms UTILIZATION false 5 min
Rpc Ops Avg Processing Time Max RpcOpsAvgProcessingTimeMax ms UTILIZATION false 5 min
Rpc Ops Avg Processing Time Min RpcOpsAvgProcessingTimeMin ms UTILIZATION false 5 min
Rpc Ops Avg Queue Time RpcOpsAvgQueueTime ms UTILIZATION true 5 min
Rpc Ops Avg Queue Time Max RpcOpsAvgQueueTimeMax ms UTILIZATION false 5 min
Rpc Ops Avg Queue Time Min RpcOpsAvgQueueTimeMin ms UTILIZATION false 5 min
Rpc Ops Number RpcOpsNumber none UTILIZATION false 5 min

Configuration help:

Hadoop Name Node Stats service

Log Track Supported: No
Config Track Supported: No
Custom Properties Supported: None
Supported Control Actions: None
Supported Metrics:

Name Alias Units Category Default On Default Interval
Availability Availability percentage AVAILABILITY true 10 min
Block Report Average Time BlockReportAverageTime ms UTILIZATION true 5 min
Block Report Max Time BlockReportMaxTime ms UTILIZATION false 5 min
Block Report Min Time BlockReportMinTime ms UTILIZATION false 5 min
Block Report Num BlockReportNum none UTILIZATION false 10 min
Block Report Num per Minute BlockReportNum1m none UTILIZATION false 10 min
Journal Transaction Average Time JournalTransactionAverageTime ms UTILIZATION true 5 min
Journal Transaction Num JournalTransactionNum none UTILIZATION false 10 min
Journal Transaction Num per Minute JournalTransactionNum1m none UTILIZATION false 10 min
Journal Transaction Max Time JournalTransactionMaxTime ms UTILIZATION false 5 min
Journal Transaction Min Time JournalTransactionMinTime ms UTILIZATION false 5 min
Journal Sync Average Time JournalSyncAverageTime ms UTILIZATION false 5 min
Journal Sync Max Time JournalSyncMaxTime ms UTILIZATION false 5 min
Journal Sync Min Time JournalSyncMinTime ms UTILIZATION false 5 min
Journal Sync Num JournalSyncNum none UTILIZATION false 10 min
Journal Sync Num per Minute JournalSyncNum1m none UTILIZATION false 10 min
Safemode Time SafemodeTime ms UTILIZATION false 5 min
FS Image Load Time FSImageLoadTime ms UTILIZATION false 5 min
Num Files Created NumFilesCreated none UTILIZATION false 10 min
Num Files Created per Minute NumFilesCreated1m none UTILIZATION false 10 min
Num Files Listed NumFilesListed none UTILIZATION false 10 min
Num Files Listed per Minute NumFilesListed1m none UTILIZATION false 10 min
Num Get Listing Ops NumGetListingOps none UTILIZATION false 10 min
Num Get Listing Ops per Minute NumGetListingOps1m none UTILIZATION false 10 min
Num Create File Ops NumCreateFileOps none UTILIZATION false 10 min
Num Create File Ops per Minute NumCreateFileOps1m none UTILIZATION false 10 min
Num Delete File Ops NumDeleteFileOps none UTILIZATION false 10 min
Num Delete File Ops per Minute NumDeleteFileOps1m none UTILIZATION false 10 min
Num Add Block Ops NumAddBlockOps none UTILIZATION false 10 min
Num Add Block Ops per Minute NumAddBlockOps1m none UTILIZATION false 10 min
Num Get Block Locations NumGetBlockLocations none UTILIZATION false 10 min
Num Get Block Locations per Minute NumGetBlockLocations1m none UTILIZATION false 10 min
Num Files Renamed NumFilesRenamed none UTILIZATION false 10 min
Num Files Renamed per Minute NumFilesRenamed1m none UTILIZATION false 10 min

Configuration help:

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.