PIG brings the Hadoop project close to developers and business users and it's used by Perl like language allowing query execution over data stored on a Hadoop cluster. It has other applications too and it's largely used for changing data like location-based data from weather or traffic sensors, web-based or social media data or machine-to-machine transactional data. Hadoop is an open source platform for handling Big Data which can work with multiple data sources. It converts any input dataset into different set of value pairs while reducing set of tuples. Its two tasks are Map task and Reduce task. It's a programming paradigm which allows massive job execution scalability against thousands of servers or clusters of servers. Presto is an open source distributed SQL query engine and Phoenix is an open source SQL query engine for Apache Hbase.
Apache Drill is an Apache incubation project and it's designed for scalability.
Shark is a data warehouse system which supports Hive's query language. It helps in making queries and it also manages large datasets in distributed storage. This technology includes Apache Hive, Shark, Apache Drill, Presto and Phoenix among many others. This database focuses on storage and retrieval of large volumes of unstructured, semi-structured or even structured data. This database category includes key-value stores and document stores. Schema-less databases, or NoSQL databases:
Spring XD supports streams for event driven data while SQLStream provides a distributed stream processing platform for streaming analytics, visualization and continuous integration of machine data.ģ. DataTorrent is a real-time streaming platform which enables businesses to perform data processing. Spark is a data processing platform which is compatible with Hadoop. Apache Storm is an open source distributed real-time computation system which simplifies streams of data and real-time processing. There are several projects in this section including Storm, Spark, Data Torrent, Spring XD and SQL Stream. It allows huge data compression and faster query times. The new column-oriented databases store data and focuses on columns and not rows. Traditional databases are excellent in online transaction processing but when it comes to query performance while data volumes grow, these databases fall short on performance. It might just be the detail that gets you hired.įinally, keep learning, especially when you feel discouraged. That being said, don’t just rely on recruitment agencies, show some initiative and contact companies on your own too. If you tell a recruiter (agent) that you’d like to go to the interview even if just for the sake of the experience, they’ll respect you for that, and will do their best to land you an interview. That being said, asking has its place especially in a team-based environment or if the deadlines are (as they often are) very tight. Putting in even hours or days for finding the solution will always be more rewarding in the long-run than asking a question on StackOverflow waiting to be spoon-fed the answer. Research before asking questions and know when to ask.
#New tech tms & business intelligence platform for free
Teach others and code forward ( code forward is a concept I came up with and it comes from “pay it forward” it basically means do a few projects for free once in a while for people who deserve it ).Īccept failure as a necessary step in self-betterment. Consequently the same has to happen to your knowledge and skill-set. If your brain doesn’t get “turned on” by new concepts, libraries, programming languages, you should not be doing it. If you’re offered a project you know nothing about, take it, you’ll learn after.īe passionate about it. I’ve been only for about 2+ years in this industry and I could already write tens of pages about what should and should not be done by aspiring developers, but here’s a few: Is there any advice you’d like to share with new students who are aspiring developers? I can honestly say that besides technical knowledge, Treehouse also gave me a certain mindset, and while I generally am a very ambitious person, with each completed course I felt energized to better myself even more. Learning with Treehouse for only 30 minutes a day can teach you the skills needed to land the job that you’ve been dreaming about.