Getty ImagesA former Indian Army soldier killed his wife by hitting her with a Buddha statue at their house in Bengaluru on March 14. The 59-year-old had served in the Indian Army for 20 years and was presently working as a conductor with Bangalore Metropolitan Transport Corporation.Javare Gowda’s younger son Chandan has lodged a complaint with the police. Chandan repeatedly called his mother Manjula on March 14 and when she didn’t respond, he lodged a complaint with the police. He told the police that his parents fight because of his father’s drinking problem, which may have led to the murder.Chandan said that his father developed alcohol addiction after retirement. Gowda would often ask his wife for money to buy alcohol. “On March 14, Manjula refused to pay him money for buying alcohol. In a fit of rage, Gowda took an iron Buddha statue and hit Manjula on her head. She died at the hospital,” the police said.Gowda had earlier taken a leave from BMTC and was staying at home since February. The police officials said that Manjula had spoken with her son on the day of the incident.Their elder son Chetan, who is a software engineer, lives abroad. The police have arrested the ex-serviceman.Crime rate in Bengaluru down?As per official records with Bengaluru Police, there has been a considerable decline in criminal activities since 2016. Around 200 murder cases were reported in Bengaluru in 2018 as compared to 228 and 234 such cases in 2016 and 2017, respectively. In a majority of the cases, the police claimed to have detained the murderers. Gangster Lakshmana murdered by rivalry gang near ISKCON templetwitterAround 500 cases of cruelty by husband were registered by Bengaluru Police in 2016, which dropped down to 375 in 2018.
Mission Mangal and Batla House day 4 box office collectionYouTube ScreenshotMission Mangal has had a terrific first weekend at the box office with its collection going close to Rs 100 crore on its day 4. On the other side, Batla House too performed well at the commercial circuit, and inched close to Rs 50 crore by the end of SundayAkshay Kumar starrer Mission Mangal has had an excellent start at the Indian box office with a collection of Rs 29.16 crore on the holiday of Independence Day. It became Akshay’s biggest opener of all time.The film had faced a major decline in its earning on second day as the collection was almost half of the opening day business. Mission Mangal had collected Rs 17.28 crore on second day, but picked up strongly on Saturday. The film had earned Rs 23.58 crore on third day at the domestic market.Nonetheless, Mission Mangal enjoyed huge occupancy in theatres across the country on its day 4, and witnessed an impressive collection. The movie collected Rs 27.54 crore at the Indian box office on Sunday, taking its total earning close to over Rs 100 crore by the end of the first weekend.While the movie was just short of Rs 2.44 crore to reach the first milestone, Mission Mangal crossed the mark by Monday afternoon.On the other hand, John Abraham starrer Batla House too has been doing well at the ticket counters. The film had started with a collection of Rs 15.55 crore, followed by a decline on second day as it earned Rs 8.84 crore. However, it also witnessed a jump in business on Saturday as it collected Rs 10.90 crore on day 3. Batla House witnessed further growth in collection on Sunday as the film collected Rs 12.70 crore on day 4, inching close to Rs 50 crore.It will be interesting to see how the two films will perform over the weekdays.
With the elections around the bend, the government will be apprising the people of the rural areas about its achievements in the development sector over the past several years. And, as usual, public money will be used for the purpose.As part of this campaign, the directorate of mass communication, under the information ministry, has taken up a project for strengthening publicity for the development of rural communities, at a cost of around Tk 600 million. The Executive Committee of the National Economic Council (ECNEC) approved the project on 3 April.This project includes campaigning on the development achievements of the government over the last few years, aimed at giving the ruling party an upper hand in the election race. Under the project videos on development will be projected in every union. This will be accompanied by concerts, women’s meetings and free meals.Director of the mass communication directorate Jasim Uddin has said the project will be implemented fully from July.The main programme is entitled ‘Egiye Jachhe Bangladesh’ (Bangladesh is advancing).The 10 initiatives of the prime minister being highlighted under the project are ‘Ekti Bari Ekti Khamar’ (one farm for every household), Asrayan, Digital Bangladesh, education assistance programme, electricity for all, community clinics and child development, social security and more.Achievements of the government at home and abroad over the last eight years will also be highlighted in the campaign through film shows, folksong sessions, women gatherings, Facebook posts, YouTube, as well as radio and TV broadcasts.The project is of three years, spanning from November 2017 to November 2020. According to the project documents, Tk 535 million or 90 per cent of the total budget will be spent between 2 and 19 of June.Commenting on the relevance of the project, former advisor of the caretaker government AB Mirza Azizul Islam said there is no justified basis for such projects. No development is achieved through such projects. He added, such projects are taken up with the election in mind, to win votes from the people.Taking up projects before the polls is nothing new.In the current fiscal, three projects have been passed for constructing schools, colleges, mosques and temples, according to the demand of the concerned MPs. Three more such projects are in the pipeline, awaiting allocations for constructing madrasas, public toilets and market places in other relevant constituencies.The MPs of these constituencies had been allocated Tk 30 million to 50 million for road construction in their areas over the last eight years. The development campaign project is the latest inclusion.Films on the development of every union will be presented. A thousand leaflets on development will be distributed and over four and a half million leaflets will be printed at a cost of Tk 100 million.Jasim Uddin went on to say, before the commencement of the project, appointments will be made, premises will be rented and other preparations will be completed.LED screens will be set up on pickup trucks in each of the 4,554 unions to screen the films.A school compound from each union will be selected for the film shows. And 20 teams will be employed to screen the films across the country. Every day at least one show will be held, at least 20 shows in each district per month.In all, 21,360 shows will be screened. Five films costing Tk 100,000 each will be produced. Two-thirds of the total budget, that is Tk 380 million, has been allocated for this purpose.Local popular folk music like bhawaiya, gombhira, jari-sari will also be performed.A total of 9,792 events will be held across the country. The events will cost Tk 20 million. Each of the folk singer will be paid Tk 700 per programme, bringing the total cost to Tk 30 million for this segment.A woman gathering will be held at each upazila. Budget for the banner, stage decoration, participators is Tk 15,000.And those watching the development campaign films will be treated to good food, with Tk 20 million allocated for their treat.AB Mirza Azizul Islam said, people can see development for themselves if it takes place. It doesn’t require further campaigning. Moreover, there is the TV and the radio for broadcasting development. Local leaders involved with the ruling party will benefit from this project, he observed.*This report, originally published in Prothom Alo print edition, has been rewritten in English by Nusrat Nowrin.
Kolkata: Six Trinamool Congress workers were killed and several were left injured when attacked by supporters of the Opposition political parties on the day of the Panchayat polls on Monday. It may be mentioned that 14 Trinamool Congress workers were killed during pre-poll clashes and on the day of the election, supporters and workers of the party were attacked at different parts of the state.Partha Chatterjee, Secretary General of Trinamool Congress, said workers and supporters of the party all across the state have shown tremendous endurance despite facing gruesome aggression. Also Read – Heavy rain hits traffic, flights”Most of the people killed on Monday were workers of the Trinamool Congress. Six TMC workers were killed,” he said.It was a few hours after the poll had started that a Trinamool Congress worker raised his voice finding attempts of rigging at a polling booth at Meriganj near Kultali in South 24-Parganas. The allegation was made against the CPI(M) that they have killed the Trinamool Congress worker, identified as Arif Ali Gazi. He was shot dead from a point-blank range. Also Read – Speeding Jaguar crashes into Merc, 2 B’deshi bystanders killedThe incident led to tension in the area. Police went to the spot and brought the situation under control. Locals protested against the incident and demanded immediate arrest of the accused.Another worker of the party — Sanjit Pramanik was killed at Shantipur in Nadia. Sanjit was an MA student and was beaten up mercilessly along with two of his friends. Before he could have understood anything, several people began beating them up. They continued thrashing him up till he fell on the ground. Even bombs were hurled at him. Police went to the spot and took them to Shantipur State General Hospital where Sanjit succumbed to his injuries. The miscreants hurled bombs to flee the area realising that they would get caught as police were approaching the spot.Bhola Tapadar, a TMC worker, was shot dead at Nakashipara in Nadia. He was taken on to the terrace of a house when he was returning after giving his vote and he was shot dead from a point blank range. The victim’s family members alleged that CPI(M) workers were behind the murder of Bhola.In another incident, Krishnapada Sarkar was killed at Tehatta in Nadia during a clash that broke out near a polling station in the area. Police picket has been posted in the area to ensure that law and order situation doesn’t deteriorate in the area.Moreover, several Trinamool Congress workers suffered injuries as well.
Kolkata: All the technical institutions and engineering colleges in the country will have to get accreditation in the next four years, said Prof Anil Dattatraya Sahasrabudhe, Chairman of the All India Council for Technical Education (AICTE) said.He was speaking on the sideline of “National Conference on Indian Higher Education: Quality Assurance, Accreditation and Ranking” organised by Education Promotion Society for India (EPSI) in a city hotel on Sunday. Also Read – Rain batters Kolkata, cripples normal lifeAICTE is the statutory body and a national-level council for technical education, under the Ministry of Human Resource Development.Prof Sahasrabudhe said accreditation has been made mandatory for the institutions. “We have given four years’ time to the colleges who are yet to get their accreditation. Many of the institutions are not yet prepared. Hence, the whole process of getting the accreditation will be completed in phases,” the AICTE chairman told the reporters. Also Read – Speeding Jaguar crashes into Mercedes car in Kolkata, 2 pedestrians killedHe also added: “Currently, around 15 percent of the institutions have got accreditation, while another 50 percent of the colleges must complete the process within the next two years. Hundred percent accreditation will be done within the next four years. The institutions will not get any extra facilities and funds will not also be provided to them if they do not complete the process of getting accreditation.”Prof Sahasrabudhe laid great emphasis on the quality of teaching and research. If the teachers are motivated and well trained, the quality of education will increase in leaps and bounds, the Chairman maintained. He also mentioned that four teachers training academies will be set up in Gujarat, Rajasthan, Kerala and Guwahati where teachers from various engineering colleges will be imparted training.The whole training would be divided into eight different modules. The programme will begin from the next year.Under Marga Darshan scheme introduced by the Center, various well performing institutions would be given funds to support otherneighbouring institutions helping them to enhance their infrastructure and other aspects.Current expenditure on higher education is 1.2 percent of the total GDP.
The separation of the directory structure is for the purpose of a clean separation of the HDFS block separation and to keep the configurations as simple as possible. This also allows us to do a proper maintenance. Now, let’s move towards changing the setup for hdfs; the file location will be /u/HBase B/hadoop-2.2.0/etc/hadoop/hdfs-site.xml. Add these properties in hdfs-site.xml: For NameNode: dfs.name.dir dfs.data.dir HBase is inspired by the Google big table architecture, and is fundamentally a non-relational, open source, and column-oriented distributed NoSQL. Written in Java, it is designed and developed by many engineers under the framework of Apache Software Foundation. Architecturally it sits on Apache Hadoop and runs by using Hadoop Distributed File System (HDFS) as its foundation. It is a column-oriented database, empowered by a fault-tolerant distributed file structure known as HDFS. In addition to this, it also provides very advanced features, such as auto sharding, load-balancing, in-memory caching, replication, compression, near real-time lookups, strong consistency (using multi-version). It uses the latest concepts of block cache and bloom filter to provide faster response to online/real-time request. It supports multiple clients running on heterogeneous platforms by providing user-friendly APIs. In this tutorial, we will discuss how to effectively set up mid and large size HBase cluster on top of Hadoop/HDFS framework. We will also help you set up HBase on a fully distributed cluster. For cluster setup, we will consider REH (RedHat Enterprise-6.2 Linux 64 bit); for the setup we will be using six nodes. This article is an excerpt taken from the book ‘HBase High Performance Cookbook’ written by Ruchir Choudhry. This book provides a solid understanding of the HBase basics. Let’s get started! Configuring and deploying Hbase Before we start HBase in fully distributed mode, we will be setting up first Hadoop-2.2.0 in a distributed mode, and then on top of Hadoop cluster we will set up HBase because HBase stores data in HDFS. Getting Ready The first step will be to create a directory at user/u/HBase B and download the tar file from the location given later. The location can be local, mount points or in cloud environments; it can be block storage: wget wget –b http://apache.mirrors.pair.com/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.tar.gz This –b option will download the tar file as a background process. The output will be piped to wget-log. You can tail this log file using tail -200f wget-log. Untar it using the following commands: tar -xzvf hadoop-2.2.0.tar.gz This is used to untar the file in a folder hadoop-2.2.0 in your current diectory location. Once the untar process is done, for clarity it’s recommended use two different folders one for NameNode and other for DataNode. I am assuming app is a user and app is a group on a Linux platform which has access to read/write/execute access to the locations, if not please create a user app and group app if you have sudo su – or root/admin access, in case you don’t have please ask your administrator to create this user and group for you in all the nodes and directorates you will be accessing. To keep the NameNodeData and the DataNodeData for clarity let’s create two folders by using the following command, inside /u/HBase B: Mkdir NameNodeData DataNodeData NameNodeData will have the data which is used by the name nodes and DataNodeData will have the data which will be used by the data nodes: ls –ltr will show the below results.drwxrwxr-x 2 app app 4096 Jun 19 22:22 NameNodeDatadrwxrwxr-x 2 app app 4096 Jun 19 22:22 DataNodeData-bash-4.1$ pwd/u/HBase B/hadoop-2.2.0-bash-4.1$ ls -ltrtotal 60Kdrwxr-xr-x 2 app app 4.0K Mar 31 08:49 bindrwxrwxr-x 2 app app 4.0K Jun 19 22:22 DataNodeDatadrwxr-xr-x 3 app app 4.0K Mar 31 08:49 etc The steps in choosing Hadoop cluster are: Hardware details required for it Software required to do the setup OS required to do the setup Configuration steps HDFS core architecture is based on master/slave, where an HDFS cluster comprises of solo NameNode, which is essentially used as a master node, and owns the accountability for that orchestrating, handling the file system, namespace, and controling access to files by client. It performs this task by storing all the modifications to the underlying file system and propagates these changes as logs, appends to the native file system files, and edits. SecondaryNameNode is designed to merge the fsimage and the edits log files regularly and controls the size of edit logs to an acceptable limit. In a true cluster/distributed environment, it runs on a different machine. It works as a checkpoint in HDFS. We will require the following for the NameNode: Components Details Used for nodes/systems Operating System Redhat-6.2 Linux x86_64 GNU/Linux, or other standard linux kernel. All the setup for Hadoop/HBase and other components used Hardware /CPUS 16 to 32 CPU cores NameNode/Secondary NameNode 2 quad-hex-/octo-core CPU DataNodes Hardware/RAM 128 to 256 GB, In special caes 128 GB to 512 GB RAM NameNode/Secondary NameNodes 128 GB -512 GB of RAM DataNodes Hardware/storage It’s pivotal to have NameNode server on robust and reliable storage platform as it responsible for many key activities like edit-log journaling. As the importance of these machines are very high and the NameNodes plays a central role in orchestrating everything,thus RAID or any robust storage device is acceptable. NameNode/Secondary Namenodes 2 to 4 TB hard disk in a JBOD DataNodes RAID is nothing but a random access inexpensive drive or independent disk. There are many levels of RAID drives, but for master or a NameNode, RAID 1 will be enough. JBOD stands for Just a bunch of Disk. The design is to have multiple hard drives stacked over each other with no redundancy. The calling software needs to take care of the failure and redundancy. In essence, it works as a single logical volume: Before we start for the cluster setup, a quick recap of the Hadoop setup is essential with brief descriptions. How to do it Let’s create a directory where you will have all the software components to be downloaded: For the simplicity, let’s take it as /u/HBase B. Create different users for different purposes. The format will be as follows user/group, this is essentially required to differentiate different roles for specific purposes: Hdfs/hadoop is for handling Hadoop-related setupYarn/hadoop is for yarn related setupHBase /hadoopPig/hadoopHive/hadoopZookeeper/hadoopHcat/hadoop Set up directories for Hadoop cluster. Let’s assume /u as a shared mount point. We can create specific directories that will be used for specific purposes. Please make sure that you have adequate privileges on the folder to add, edit, and execute commands. Also, you must set up password less communication between different machines like from name node to the data node and from HBase master to all the region server nodes. Once the earlier-mentioned structure is created; we can download the tar files from the following locations: -bash-4.1$ ls -ltrtotal 32drwxr-xr-x 9 app app 4096 hadoop-2.2.0drwxr-xr-x 10 app app 4096 zookeeper-3.4.6drwxr-xr-x 15 app app 4096 pig-0.12.1drwxrwxr-x 7 app app 4096 HBase -0.98.3-hadoop2drwxrwxr-x 8 app app 4096 apache-hive-0.13.1-bindrwxrwxr-x 7 app app 4096 Jun 30 01:04 mahout-distribution-0.9 You can download these tar files from the following location: wget –o https://archive.apache.org/dist/HBase /HBase -0.98.3/HBase -0.98.3-hadoop1-bin.tar.gzwget -o https://www.apache.org/dist/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gzwget –o https://archive.apache.org/dist/mahout/0.9/mahout-distribution-0.9.tar.gzwget –o https://archive.apache.org/dist/hive/hive-0.13.1/apache-hive-0.13.1-bin.tar.gzwget -o https://archive.apache.org/dist/pig/pig-0.12.1/pig-0.12.1.tar.gz Here, we will list the procedure to achieve the end result of the recipe. This section will follow a numbered bullet form. We do not need to give the reason that we are following a procedure. Numbered single sentences would do fine. Let’s assume that there is a /u directory and you have downloaded the entire stack of software from: /u/HBase B/hadoop-2.2.0/etc/hadoop/ and look for the file core-site.xml. Place the following lines in this configuration file: fs.default.name hdfs://addressofbsdnsofmynamenode-hadoop:9001 You can specify a port that you want to use, and it should not clash with the ports that are already in use by the system for various purposes. Save the file. This helps us create a master /NameNode. Now, let’s move to set up SecondryNodes, let’s edit /u/HBase B/hadoop-2.2.0/etc/hadoop/ and look for the file core-site.xml: fs.defaultFS hdfs://custome location of your hdfs fs.checkpoint.dir /u/HBase B/dn001/hadoop/hdf/secdn /u/HBase B/dn002/hadoop/hdfs/secdn We can go for the https setup for the NameNode too, but let’s keep it optional for now: Let’s set up the yarn resource manager: Let’s look for Yarn setup: /u/HBase B/hadoop-2.2.0/etc/hadoop/ yarn-site.xml For resource tracker a part of yarn resource manager: yarn.yourresourcemanager.resourcetracker.addressyouryarnresourcemanager.full.hostname:8025 For resource schedule part of yarn resource scheduler: yarn.yourresourcemanager.scheduler.addressyourresourcemanager.full.hostname:8030 For scheduler address: yarn.yourresourcemanager.addressyourresourcemanager.full.hostname:8050 For scheduler admin address: yarn.yourresourcemanager.admin.addressyourresourcemanager.full.hostname:8041 To set up a local dir: yarn.yournodemanager.local-dirs /u/HBase /dnn01/hadoop/hdfs /yarn,/u/HBase B/dnn02/hadoop/hdfs/yarn To set up a log location: yarn.yournodemanager.logdirs /u/HBase B/var/log/hadoop/yarn This completes the configuration changes required for Yarn. Now, let’s make the changes for Map reduce: Let’s open the mapred-site.xml: /u/HBase B/hadoop-2.2.0/etc/hadoop/mapred-site.xml Now, let’s place this property configuration setup in the mapred-site.xml and place it between the following: mapreduce.yourjobhistory.addressyourjobhistoryserver.full.hostname:10020 Once we have configured Map reduce job history details, we can move on to configure HBase . Let’s go to this path /u/HBase B/HBase -0.98.3-hadoop2/conf and open HBase -site.xml. You will see a template having the following: We need to add the following lines between the starting and ending tags: HBase .rootdirhdfs://HBase .yournamenode.full.hostname:8020/apps/HBase /dataHBase .yourmaster.info.bindAddress$HBase .yourmaster.full.hostname This competes the HBase changes. ZooKeeper: Now, let’s focus on the setup of ZooKeeper. In distributed env, let’s go to this location and rename the zoo_sample.cfg to zoo.cfg: /u/HBase B/zookeeper-3.4.6/conf Open zoo.cfg by vi zoo.cfg and place the details as follows; this will create two instances of zookeeper on different ports: yourzooKeeperserver.1=zoo1:2888:3888yourZooKeeperserver.2=zoo2:2888:3888 If you want to test this setup locally, please use different port combinations. In a production-like setup as mentioned earlier, yourzooKeeperserver.1=zoo1:2888:3888 is server.id=host:port:port: yourzooKeeperserver.1= server.idzoo1=host2888=port3888=port Atomic broadcasting is an atomic messaging system that keeps all the servers in sync and provides reliable delivery, total order, casual order, and so on. Region servers: Before concluding it, let’s go through the region server setup process. Go to this folder /u/HBase B/HBase -0.98.3-hadoop2/conf and edit the regionserver file. Specify the region servers accordingly: RegionServer1 RegionServer2 RegionServer3 RegionServer4 RegionServer1 equal to the IP or fully qualified CNAME of 1 Region server. You can have as many region servers (1. N=4 in our case), but its CNAME and mapping in the region server file need to be different. Copy all the configuration files of HBase and ZooKeeper to the relative host dedicated for HBase and ZooKeeper. As the setup is in a fully distributed cluster mode, we will be using a different host for HBase and its components and a dedicated host for ZooKeeper. Next, we validate the setup we’ve worked on by adding the following to the bashrc, this will make sure later we are able to configure the NameNode as expected: It preferred to use it in your profile, essentially /etc/profile; this will make sure the shell which is used is only impacted. Now let’s format NameNode: Sudo su $HDFS_USER/u/HBase B/hadoop-2.2.0/bin/hadoop namenode -format HDFS is implemented on the existing local file system of your cluster. When you want to start the Hadoop setup first time you need to start with a clean slate and hence any existing data needs to be formatted and erased. Before formatting we need to take care of the following. Check whether there is a Hadoop cluster running and using the same HDFS; if it’s done accidentally all the data will be lost. /u/HBase B/hadoop-2.2.0/sbin/hadoop-daemon.sh –config$HADOOP_CONF_DIR start namenode Now let’s go to the SecondryNodes: Sudo su $HDFS_USER/u/HBase B/hadoop-2.2.0/sbin/hadoop-daemon.sh –config $HADOOP_CONF_DIR start secondarynamenode Repeating the same procedure in DataNode: Sudo su $HDFS_USER/u/HBase B/hadoop-2.2.0/sbin/hadoop-daemon.sh –config $HADOOP_CONF_DIR start datanodeTest 01> See if you can reach from your browser http://namenode.full.hostname:50070: Test 02> sudo su $HDFS_USER touch /tmp/hello.txt Now, hello.txt file will be created in tmp location: /u/HBase B/hadoop-2.2.0/bin/hadoop dfs -mkdir -p /app/u/HBase B/hadoop-2.2.0/bin/hadoop dfs -mkdir -p /app/apphduserThis will create a specific directory for this application user in the HDFS FileSystem location(/app/apphduser)/u/HBase B/hadoop-2.2.0/bin/hadoop dfs -copyFromLocal /tmp/hello.txt /app/apphduser/u/HBase B/hadoop-2.2.0/bin/hadoop dfs –ls /app/apphduser apphduser is a dirctory which is created in hdfs for a specific user. So that the data is sepreated based on the users, in a true production env many users will be using it. You can also use hdfs dfs –ls / commands if it shows hadoop command as depricated. You must see hello.txt once the command executes: Test 03> Browse http://datanode.full.hostname:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=/&nnaddr=$datanode.full.hostname:8020 It is important to change the data host name and other parameters accordingly. You should see the details on the DataNode. Once you hit the preceding URL you will get the following screenshot: On the command line it will be as follows: Validate Yarn/MapReduce setup and execute this command from the resource manager: /u/HBase B/hadoop-2.2.0/sbin/yarn-daemon.sh –config $HADOOP_CONF_DIR start resourcemanager Execute the following command from NodeManager: /u/HBase B/hadoop-2.2.0/sbin /yarn-daemon.sh –config$HADOOP_CONF_DIR start nodemanager Executing the following commands will create the directories in the hdfs and apply the respective access rights: Cd u/HBase B/hadoop-2.2.0/binhadoop fs -mkdir /app-logs // creates the dir in HDFShadoop fs -chown $YARN_USER /app-logs //changes the ownershiphadoop fs -chmod 1777 /app-logs // explained in the note sectionExecute MapReduce Start jobhistory servers: /u/HBase B/hadoop-2.2.0/sbin/mr-jobhistory-daemon.sh start historyserver –config $HADOOP_CONF_DIR Let’s have a few tests to be sure we have configured properly: Test 01: From the browser or from curl use the link to browse: http://yourresourcemanager.full.hostname:8088/. Test 02: Sudo su $HDFS_USER/u/HBase B/hadoop-2.2.0/bin/hadoop jar /u/HBase B/hadoop-2.2.0/hadoop-mapreduce/hadoop-mapreduce-examples-126.96.36.199-alpha.jar teragen 100 /test/10gsort/input/u/HBase B/hadoop-2.2.0/bin/hadoop jar /u/HBase B/hadoop-2.2.0/hadoop-mapreduce/hadoop-mapreduce-examples-188.8.131.52-alpha.jar Validate the HBase setup: Login as $HDFS_USER/u/HBase B/hadoop-2.2.0/bin/hadoop fs –mkdir -p /apps/HBase/u/HBase B/hadoop-2.2.0/bin/hadoop fs –chown app:app –R /apps/HBase Now login as $HBase _USER: /u/HBase B/HBase -0.98.3-hadoop2/bin/HBase -daemon.sh –-config $HBase _CONF_DIR start master This command will start the master node. Now let’s move to HBase Region server nodes: /u/HBase B/HBase -0.98.3-hadoop2/bin/HBase -daemon.sh –-config $HBase _CONF_DIR start regionserver This command will start the regionservers: For a single machine, direct sudo ./HBase master start can also be used. Please check the logs in case of any logs at this location /opt/HBase B/HBase -0.98.5-hadoop2/logs. You can check the log files and check for any errors: Now let’s login using: Sudo su- $HBase _USER/u/HBase B/HBase -0.98.3-hadoop2/bin/HBase shell We will connect HBase to the master. Validate the ZooKeeper setup. If you want to use an external zookeeper, make sure there is no internal HBase based zookeeper running while working with the external zookeeper or existing zookeeper and is not managed by HBase : For this you have to edit /opt/HBase B/HBase -0.98.5-hadoop2/conf/ HBase -env.sh. Change the following statement (HBase _MANAGES_ZK=false): # Tell HBase whether it should manage its own instance of Zookeeper or not. export HBase _MANAGES_ZK=true. Once this is done we can add zoo.cfg to HBase ‘s CLASSPATH. HBase looks into zoo.cfg as a default lookup for configurations dataDir=/opt/HBase B/zookeeper-3.4.6/zooData # this is the place where the zooData will be present server.1=172.28.182.45:2888:3888 # IP and port for server 01 server.2=172.29.75.37:4888:5888 # IP and port for server 02 You can edit the log4j.properties file which is located at /opt/HBase B/zookeeper-3.4.6/conf and point the location where you want to keep the logs. # Define some default values that can be overridden by system properties: zookeeper.root.logger=INFO, CONSOLEzookeeper.console.threshold=INFOzookeeper.log.dir=.zookeeper.log.file=zookeeper.logzookeeper.log.threshold=DEBUGzookeeper.tracelog.dir=. # you can specify the location herezookeeper.tracelog.file=zookeeper_trace.log Once this is done you start zookeeper with the following command: -bash-4.1$ sudo /u/HBase B/zookeeper-3.4.6/bin/zkServer.sh startStarting zookeeper … STARTED You can also pipe the log to the ZooKeeper logs: /u/logs//u/HBase B/zookeeper-3.4.6/zoo.out 2>&1 2 : refers to the second file descriptor for the process, that is stderr. > : means re-direct&1: means the target of the rediretion should be the same location as the first file descriptor i.e stdout How it works Sizing of the environment is very critical for the success of any project, and it’s a very complex task to optimize it to the needs. We dissect it into two parts, master and slave setup. We can divide it in the following parts: Master-NameNodeMaster-Secondary NameNodeMaster-JobtrackerMaster-Yarn Resource ManagerMaster-HBase MasterSlave-DataNodeSlave-Map Reduce TasktrackerSlave-Yarn Node ManagerSlave-HBase Region server NameNode: The architecture of Hadoop provides us a capability to set up a fully fault tolerant/high availability Hadoop/HBase cluster. In doing so, it requires a master and slave setup. In a fully HA setup, nodes are configured in active passive way; one node is always active at any given point of time and the other node remains as passive. Active node is the one interacting with the clients and works as a coordinator to the clients. The other standby node keeps itself synchronized with the active node and to keep the state intact and live, so that in case of failover it is ready to take the load without any downtime. Now we have to make sure that when the passive node comes up in the event of a failure, the passive node is in perfect sync with the active node, which is currently taking the traffic. This is done by Journal Nodes(JNs), these Journal Nodes use daemon threads to keep the primary and sercodry in perfect sync. Journal Node: By design, JournalNodes will only have single NameNode acting as a active/primary to be a writer at a time. In case of failure of the active/primary, the passive NameNode immediately takes the charge and transforms itself as active, this essentially means this newly active node starts writing to Journal Nodes. Thus it totally avoids the other NameNode to stay in active state, this also acknowledges that the newly active node work as a fail over node. JobTracker: This is an integral part of Hadoop EcoSystem. It works as a service which farms MapReduce task to specific nodes in the cluster. ResourceManager (RM): This responsibility is limited to scheduling, that is, only mediating available resources in the system between different needs for the application like registering new nodes, retiring dead nodes, it dose it by constantly monitoring the heartbeats based on the internal configuration. Due to this core design practice of explicit separation of responsibilities and clear orchestrations of modularity and with the inbuilt and robust scheduler API, This allows the resource manager to scale and support different design needs at one end, and on the other, it allows us to cater to different programming models. HBase Master: The Master server is the main orchestrator for all the region servers in the HBase cluster . Usually, it’s placed on the ZooKeeper nodes. In a real cluster configuration, you will have 5 to 6 nodes of Zookeeper. DataNode: It’s a real workhorse and does most of the heavy lifting; it runs the MapReduce Job and stores the chunks of HDFS data. The core objective of the data node was to be available on the commodity hardware and should be agnostic to the failures. It keeps some data of HDFS, and the multiple copy of the same data is sprinkled around the cluster. This makes the DataNode architecture fully fault tolerant. This is the reason a data node can have JBOD01 rather rely on the expensive RAID02. MapReduce: Jobs are run on these DataNodes in parallel as a subtask. These subtasks provides the consistent data across the cluster and stays consistent. So we learned about the HBase basics and how to configure and set it up. We set up HBase to store data in Hadoop Distributed File System. We also explored the working structure of RAID and JBOD and the differences between both filesystems. If you found this post useful, be sure to check out the book ‘HBase High Perforamnce Cookbook’ to learn more about configuring HBase in terms of administering and managing clusters as well as other concepts in HBase. Read Next Understanding the HBase Ecosystem Configuring HBase 5 Mistake Developers make when working with HBase /u/HBase B/dnn01/hadoop/hdfs/dn,/HBase B/u/dnn02/hadoop/hdfs/dn Now, let’s go for NameNode for http address or to access using http protocol: dfs.http.addressyournamenode.full.hostname:50070dfs.secondary.http.addresssecondary.yournamenode.full.hostname:50090 For DataNode: /u/HBase B/nn01/hadoop/hdfs/nn,/u/HBase B/nn02/hadoop/hdfs/nn
Tags: Bermuda, Hurricane, Hurricane Florence, Weather By: The Associated Press Share Thursday, September 6, 2018 MIAMI — Hurricane Florence flirted with Category 4 status last night before dropping back down to Category 3, however forecasters are warning the storm is still likely to cause “life-threatening” surf and rip current conditions in Bermuda later this week.The National Hurricane Center said the storm’s maximum sustained winds Wednesday afternoon are estimated to be 215 kph.Hurricane Florence is centred about 2,080 kilometres east-southeast of Bermuda and is moving northwest at 20 kph.Forecasters expect Florence to weaken somewhat over the next few days, but they anticipate it will still be a powerful hurricane through early next week.Officials say swells caused by Florence will begin to affect Bermuda on Friday.There are no coastal watches or warnings currently in place. << Previous PostNext Post >> Florence now a Category 3 hurricane in Atlantic